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CHAPTER  1 


INTRODUCTION 

1.1.  Motivation 

In  many  engineering  situations  a  variety  of  possible  measurements  can  be  carried  out 
on  a  physical  system  or  process,  and  the  question  that  naturally  arises  is  what  measure¬ 
ment  process  is  optimal.  Thus  it  may  be  necessary  to  communicate  information  through 
noisy  channels  to  an  observer,  and  a  decision  on  what  information  to  transmit  may  need 
to  be  made.  Based  on  the  observations  received,  the  controller  is  required  to  make  deci¬ 
sions  that  may  affect  the  cost  incurred  as  well  as  the  further  evolution  of  the  system. 

Consider,  for  example,  the  following  decentralized  systems: 

(a)  Control  of  a  Space  Probe  from  an  Earth  Station:  We  may  think  of  a  space  probe 
and  an  earth  station  together  constituting  a  decentralized  system.  The  sensor  aboard  the 
space  probe  is  required  to  transmit  data  over  the  downlink,  after  suitable  encoding  at 
some  permissible  power  level.  The  earth  station  may  be  required  to  command  the  space 
probe  to  move  along  a  certain  trajectory,  or  it  may  wish  to  form  estimates  of  the 
transmitted  measurement.  The  problem,  then,  is  one  of  simultaneously  designing  the 
communication  strategy  used  by  the  probe  and  the  control  policy  of  the  earth  station. 

(b)  Tracking  Systems:  In  tracking  systems  it  may  be  possible  to  allow  for 
transmitting  measurements  at  larger  power  levels  (with  an  increase  in  the  associated 
transmission  cost)  in  order  to  combat  noise  and  increase  control  efficiency.  Here  not  only 
the  communication  strategy  but  also  the  transmission  power  level  need  to  be  designed. 


Problems  requiring  simultaneous  communication  and  control  also  arise  in  many 
socioeconomic  situations,  control  of  flexible  structures,  and  other  areas  where  the  design 


or  allocation  of  limited  transmission  resources  is  important.  The  essentials  of  the  situa¬ 
tions  involve 

(i)  A  stochastic  system,  usually  dynamic  and/or  decentralized,  consisting  of  a 
number  of  decision  makers  (synonymously  agents)  along  with  a  mathematical  description 
of  their  interaction  with  the  system  and  among  themselves. 

(ii)  Elements  of  uncertainty  (noises)  entering  the  system,  the  underlying  probability 
spaces  f or  which  are  beyond  the  control  of  the  decision  makers. 

(iii)  An  information  structure,  which  characterizes  the  information  gained  and 
recalled  by  each  decision  maker. 

(iv)  A  set  of  possible  alternatives  (decisions)  for  each  agent,  and  permissible  stra¬ 
tegies  which  are  mappings  from  the  information  space  to  the  decision  space  of  each  agent. 

(v)  An  objective  functional  that  summarizes  the  preference  ordering  among  various 
alternatives  for  the  set  of  decision  makers. 

We  note  here  that  for  the  class  of  problems  described  above  there  exist  very  general 
formulations  where,  for  example,  the  order  in  which  the  agents  act  may  be  determined  by 
a  chance  mechanism  ( Witsenhausen  [1971])  or  the  various  agents  exhibit  a  divergence  of 
interests  requiring  a  different  objective  functional  for  each  agent  (Basar  and  Olsder 
[1982]).  Here  we  restrict  our  attention  to  classes  of  problems  where  in  which 

(i)  The  order  in  which  the  agents  act  is  fixed  in  advance. 

(ii)  All  noises  which  may  enter  the  system  are  zero  mean  Gaussian,  and  further 

(iii)  The  problem  involves  the  design  of  two  kinds  of  strategies: 

-  Communication  strategics  for  generating  information-bearing  signals. 
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-  Control  policies  for  forming  estimates,  minimizing  errors  and  reducing  costs. 

Many  such  problems  have  been  the  subject  of  previous  investigations.  Athans  [1972] 
considers  the  problem  of  selecting,  at  each  instant  of  time,  one  measurement  provided  by 
one  out  of  many  sensors,  with  each  measurement  ha  /ing  an  associated  cost.  For  the  class 
of  linear  stochastic  dynamic  systems  and  measurement  subsystems,  a  weighted  combina¬ 
tion  of  prediction  cost  and  accumulated  observation  cost  is  minimized,  with  the  optimal 
measurement  strategy  being  obtained  by  solving  a  deterministic  optimal  control  problem. 
A  similar  formulation  was  considered  ear.ier  by  Meier  et  al.  [  1967]  who  obtained  a  com¬ 
putational  solution  using  dynamic  programming.  Herring  and  Melsa  [1974]  generalize  the 
above  results  to  allow  the  selection,  at  each  instant  of  time,  of  the  best  combination  of 
measurement  devices  as  opposed  to  the  best  single  device,  and  Mehra  [1976]  presents 
measurement  schedules  and  sensor  designs  for  linear  stochastic  systems  subject  to  a  con¬ 
straint  on  the  total  measurement  precision  so  as  to  minimize  a  norm  of  the  error  covari¬ 
ance  matrix. 

In  contrast  to  the  above  studies,  where  the  cost  is  the  covariance  of  the  error  of  the 
Kalman  estimate,  Chu  [1978]  considers  the  problem  of  finding  the  best  measurement  for 
static  problems  with  arbitrary  quadratic  cost.  A  similar  framework  to  that  of  Chu  is 
adopted  by  Papavassilopoulos  [1983]  for  the  case  of  more  than  one  decision  maker,  with 
the  measurements  still  being  restricted  to  be  linear  in  the  state. 

Lafortune  [1985]  presents  a  general  theorem  for  the  computation  of  optimal  solution 
to  discrete  time  stochastic  control  problems  when  the  decision  makers  have  the  additional 
freedom  of  choosing  at  each  step  among  different  sets  of  observations.  Lafortune  applies 
his  resulls  to  finite  stale,  controlled  Markov  chains  as  well  as  linear  Gaussian  systems 
with  quadratic  cost  functionals.  The  control  of  Markov  chains  is  also  considered  by 
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Varaiya  and  Walrand  [1983],  who  obtain  a  complete  solution  for  a  class  of  symmetric 
channels  with  noiseless  feedback. 

Whittle  and  Rudge  [1976]  consider  a  situation  where  one  agent  observes  a  stationary 
ergodic  Gaussian  process  and  on  its  basis  forms  a  sequence,  through  not  necessarily  a 
linear  transf ormation.  The  other  agent  observes  a  noisy  replica  of  the  first  agent’s  deci¬ 
sion,  and  he  is  required  to  form  an  estimate  of  the  original  Gaussian  sequence.  This  prob¬ 
lem  of  simultaneous  communication  and  control  is  solved  using  the  concepts  of  statistical 
communication  theory,  and  the  solution  requires  that  arbitrarily  large  signal  blocks  be 
available  at  all  stages.  In  control  theoretic  terms  their  solutions  are  unrealizable,  since  the 
actions  taken  at  a  given  time  depend  also  on  observations  which  lie  in  the  future. 

The  problem  considered  by  Whittle  and  Rudge  is  somewhat  different  from  the  ones 
mentioned  earlier,  because  in  the  earlier  formulations  the  measurement  model  is  specified 
except  for  certain  parameters,  and  the  optimal  selection  of  these  parameters  has  been 
investigated.  Here,  we  shall  formulate  and  solve  a  class  of  problems  related  to  that  of 
Whittle  and  Rudge  under  the  additional  restriction  of  causality.  When  the  design  of  the 
measurement  strategy  itself  is  part  of  the  problem,  the  problem  becomes  considerably 
more  difficult,  since  the  action  of  one  decision  maker  affects  the  information  of  the  other, 
and  there  is  no  way  in  which  this  other  decision  maker  can  have  access  to  the  information 
on  which  the  first  one  acted.  Such  information  structures  have  been  called  nonclassical  in 
the  stochastic  team  literature,  in  the  next  section  we  study  different  information  struc¬ 
tures  and  the  related  computational  considerations. 


1.2.  Information  Structures  and  Computational  Considerations 

In  this  section  we  discuss  the  various  types  of  information  structures  associated  with 
stochastic  team  problems.  These  information  structures  characterize  the  precise  static  or 
dynamic  information  gained  or  recalled  by  each  decision  maker  at  each  stage  of  a  decision 
process,  and  are  customarily  distinguished  as  classical,  quasiclassical  or  nonclassical.  We 
then  discuss  some  of  the  issues  of  computational  complexity  which  arise  when  the  numer¬ 
ical  derivation  of  the  optimal  team  solution  is  considered. 

1.2.1.  Classical  and  quasiclassical  information  structures 

Classical  information  patterns  include  deterministic  patterns  and  centralized  infor¬ 
mation  patterns.  Deterministic  patterns  arise  when  the  information  is  not  noise-corrupted 
and  may  be  of  the  open-loop  type  in  which  only  the  initial  value  of  the  state  is  available 
and  no  dynamic  information  is  acquired,  or  of  the  closed-loop  type  where  perfect  informa¬ 
tion  concerning  the  current  value  of  the  state  is  also  acquired.  Centralized  patterns  arise 
when  all  agents  exchange  their  measurements  without  any  delay  and  also  recall  the  past 
information. 

Under  the  deterministic  or  centralized  stochastic  information  patterns,  stochastic 
team  problems  become  equivalent  to  stochastic  control  problems  and  the  solution  tech¬ 
niques  for  these  (e.g.,  Bertsekas  [1987],  Kumar  and  Varaiya  [1986])  are  directly  applica¬ 
ble.  Thus  for  stochastic  teams  with  classical  information  patterns,  when 

(i)  the  measurements  are  linear  in  the  primitive  random  variables  and  controls, 

(li)  the  primitive  random  variables  are  jointly  Gaussian. 

(iii)  the  cost  function  is  quadratic  in  ’he  control  vector  and  the  primitive  random 


variables,  and 


1.2.2.  Nonclassical  information  structures 

An  information  pattern  is  nonclassical  if  it  is  not  partially  nested.  Alternatively,  if 
the  decision  maker  j’s  action  affects  the  information  of  i,  and  there  is  no  way  by  which  i 
can  infer  the  information  available  to  j,  then  the  information  structure  is  said  to  be  of  the 
nonclassical  type.  Under  the  nonclassical  information  pattern,  the  derivation  of  the 
optimal  team  solution  meets  with  formidable  difficulties.  One  way  of  viewing  these 
difficulties  is  that  the  control  plays  a  triple  role  (Ho  [1980]),  viz., 

(i)  the  deterministic  control  effort  of  reducing  the  error; 

(ii)  improvement  of  future  knowledge  of  uncertainty; 

(iii)  signalling  to  agents  acting  in  the  future  some  useful  information  that  they  do 
not  necessarily  acquire; 

and  these  roles  are  in  general  conflicting. 

Classes  of  tractable  problems  with  nonclassical  information  have  been  very  difficult 
to  identify.  Witsenhausen  [  1 968]  established  that  some  of  the  simplest  linear-quadratie- 
Gaussian  (LQG)  stochastic  teams  with  nonclassical  information  do  not  admit  optimal 
linear  solutions;  in  fact  the  optimal  solution  to  the  problem  formulated  by  Witsenhausen 
is  not  yet  known.  Very  recently,  however,  some  success  has  been  reported  for  two-person 
stochastic  teams  with  nonclassical  information.  Bansal  and  Basar  [1987a]  show  that  it  is 
the  presence  of  the  product  term  between  the  decision  variables  which,  coupled  wi.h  the 
nonclassical  information,  makes  the  LQG  problem  intractable,  and  in  the  absence  of  this 
product  term  optimal  solutions  may  readily  be  found.  Some  nonclassical  multipath  sys¬ 
tems  have  also  been  shown  to  admit  optimal  linear  solutions  under  quadratic  costs  and 
Gaussian  noises  (Bansal  and  Basar  [  1987b]). 


Here  we  are  concerned  with  nonclassical  information  patterns  because  many  prob¬ 
lems  involving  the  simultaneous  design  of  communication  and  control  strategies  exhibit 
an  information  pattern  of  this  type.  Recall  that  the  third  role  of  controlling  agents  in 
nonclassical  teams  mentioned  above  is  the  role  of  "signalling*  information  to  agents  acting 
in  the  future,  and  this  role  is  absent  in  the  case  of  classical  and  quasiclassical  information 
structures.  When  explicit  design  of  the  communication  strategy  itself  is  considered,  it  is 
the  "signalling"  policy  that  is  being  designed,  and  nonclassical  patterns  arise  naturally. 

It  is  for  the  above  reasons  that  the  problem  considered  by  Whittle  and  Rudge  [1976], 
where  the  communication  strategy  is  to  be  designed,  is  significantly  more  complicated 
than  the  other  problems,  where  the  measurement  model  is  specified,  except  for  certain 
parameters,  as  in  Athans  [1972],  Herring  and  Melsa  [1974],  Mehra  [1976]. 

In  the  course  of  this  work  we  shall  identify  some  classes  of  stochastic  teams  with 
nonclassical  information  which  are  tractable  in  spite  of  the  above  mentioned  difficulties. 

1.2.3.  Computational  complexity  of  stochastic  team  problems 

The  severe  difficulty  encountered  when  the  solution  of  stochastic  team  problems  is 
attempted  via  numerical  techniques  has  triggered  some  research  into  their  complexity. 
Some  recent  results  (Papadimitriou  and  Tsitsiklis  [1982],  Tsitsiklis  and  Athans  [1985]) 
relate  the  complexity  of  stochastic  team  problems  to  that  of  known  intractable  problems. 
Using  the  tools  of  computational  complexity  the  NP  completeness  of  the  discrete  version 
of  the  static  team  problem  has  been  established,  and  some  progress  has  also  been  made  in 
defining  complexity  concepts  for  continuous  time  problems  by  relating  the  complexity  ol 
the  continuous  version  to  that  of  its  discretized  counterpart  (Papadimitriou  and  Tsitsiklis 
[1986]).  In  the  same  work  the  NP  completeness  ol  the  discrete  version  of  Witsen hausen's 
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problem  has  also  been  established,  thus  explaining  the  failures,  reported  in  literature,  to 
attack  it  computationally  (Ho  and  Chang  [1980]). 

Quite  recently,  Witsenhausen  [1988]  has  developed  a  notion  of  equivalence  between 
stochastic  control  problems  that  is  suitable  for  complexity  analysis.  He  has  shown  that  a 
large  class  of  problems  with  dynamic  information  can  be  reduced  to  equivalent  static 
problems  with  a  transformed  cost  functional,  and  this  class  includes  all  sequential 
discrete  variable  problems.  Person-by-person  optimality  is  then  a  sufficient  condition  for 
global  optimality  under  smoothness  and  convexity  conditions;  however,  when  there  is  no 
convexity  then  person-by-person  optimality  is  not  sufficient. 

While  approaches  based  on  the  theory  of  NP  completeness  shed  important  light  on 
the  applicability  (or  nonapplicability)  of  numerical  techniques  for  discretized  versions  of 
stochastic  teams,  being  a  worst  case  scenario  they  obscure  important  special  cases  which 
may  admit  relatively  simple  optimal  solutions.  For  example,  both  the  static  team  prob¬ 
lem  as  well  as  Witsenhausen ’s  counterexample  have  discretized  versions  which  are  NP 
complete,  but  the  former  is  quite  tractable  under  linear  quadratic  Gaussian  assumptions 
(Radner  [1962]),  whereas  the  latter  is  still  unsolved.  The  problem  of  carving  out  classes 
of  tractable  problems  within  the  general  class  of  nonclassical  stochastic  teams  is  an  impor¬ 
tant  one,  and  this  thesis  is  a  contribution  to  this  direction. 


1.3.  Nonclassical  Patterns  and  the  Problem  of  Information  Transmission 

Consider  the  simplest  problem  of  transmitting  information,  where  the  value  of  a 
random  variable  is  to  be  transmitted,  after  suitable  encoding,  over  a  noisy  channel.  The 
noisy  channel  output  is  available  at  a  decoder  where  it  is  used  to  construct  an  estimate  of 
the  input  variable.  In  order  to  make  this  problem  well  defined,  it  is  customary  in  the 
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communications  literature  to  assume  that  the  encoder  operates  under  a  hard  power  con¬ 
straint,  i.e.,  the  mean-square  level  at  the  output  of  the  encoder  cannot  exceed  a  certain 
predefined  level. 

This  standard  information  transmission  problem  is  depicted  in  Figure  1.1,  where  x  is 
the  input  to  be  transmitted,  u  is  the  encoder  output  satisfying  E[u2]<P2,  w  is  the  channel 


noise,  and  the  output  of  the  channel 


y  =  u  +  w 


is  used  by  the  decoder  to  form  an  estimate  of  x,  designated  by  x. 

Note  that  the  communication  problem  is  one  with  a  nonclassical  information  struc¬ 
ture,  since  the  action  of  the  encoder  affects  the  information  of  the  decoder,  but  the  decoder 
does  not  have  access  to  the  information  of  the  encoder.  (Indeed,  if  the  decoder  did  have 
access  to  the  information  of  the  encoder,  then  the  problem  would  be  trivial,  the  estimate 
of  x  exact,  and  the  channel  redundant.) 


Channel  Noise 


Figure  1.1.  The  information  transmission  problem. 


Information  Theory  provides  the  fundamental  results  for  the  analysis  of  the  infor¬ 
mation  transmission  system  of  Figure  1.1.  One  of  the  most  important  results  of  Informa¬ 
tion  Theory  relates  channel  capacity  to  the  rate  distortion  of  the  source.  (A  study  of 


rjK  rjrm  n-  r, 


channel  capacity  and  rate  distortion  may  be  found,  for  example,  in  Ash  [1965],  Berger 
[1971]  and  Gallager  [1968].) 

Let  R(D)  denote  the  rate  distortion  function,  (Berger  [1971]),  corresponding  to  a 
source  under  an  appropriate  distortion  measure  D.  Shannon  [1959]  proved  that  it  is  not 
possible  with  any  coding  scheme  to  transmit  the  source  under  consideration  through  any 
channel  of  capacity  less  than  R(D)  without  incurring  an  average  distortion  larger  than  or 
equal  to  D.  Conversely,  given  any  channel  of  capacity  C  >  R(D),  coding  schemes  exist 
(possibly  using  arbitrarily  large  block  lengths)  which  result  in  an  average  distortion  arbi¬ 
trarily  close  to  D  when  used  over  this  channel.  Thus,  R(D)  is  the  minimum  channel  capa¬ 
city  required  to  reproduce  the  given  source  at  the  decoder  with  average  distortion  at  most 
D.  For  any  channel,  the  minimum  possible  distortion  D*  that  can  result  from  its  use  is 
related  to  its  capacity  C  by 

R(D*)  =  C  . 

We  shall  find  the  above  relation  between  capacity  and  rate  distortion  quite  useful  for 
the  analysis  of  some  decentralized  stochastic  team  problems,  because  the  information 
transmission  problem  is  a  special  case  of  problems  with  nonclassical  information.  We  will 
also  need  the  concepts  of  entropy  and  conditional  entropy  (see,  for  example,  Abramson 
[1963]);  these  will  be  introduced  in  subsequent  chapters  as  required. 

1.4.  Organization  of  the  Thesis 

The  organization  of  this  thesis  is  as  follows.  In  Chapter  2  we  formulate  and  analyze 
some  fundamental  classes  of  stochastic  team  problems  with  two  decentralized  agents, 
exhibiting  a  nonclassical  information  pattern.  We  identify  instances  of  such  problems  for 
which  the  optimal  solutions  are  linear  and  may  readily  be  found  despite  the  intractability 
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of  the  general  problem. 

In  Chapter  3  we  consider  a  stochastic  dynamic  decision  problem,  where  at  ea_h  step 
two  consecutive  decisions  must  be  made,  one  being  what  information-bearing  signal  to 
transmit,  and  the  other  what  control  action  to  exert.  Such  problems  arise  in  the  simul¬ 
taneous  optimization  of  both  the  observation  and  the  control  sequences  in  stochastic  sys¬ 
tems.  This  problem  is  solved  completely  for  first-order  auto-regressive  moving  average 
(ARMA)  systems  under  the  quadratic  cost  criterion.  The  results  are  further  extended  to 
cases  where  the  time  horizon  is  infinite  and  the  cost  function  is  discounted. 

In  Chapter  4  we  consider  stochastic  dynamic  decision  problems  requiring  simultane¬ 
ous  optimization  of  both  the  observation  and  the  control  sequences  for  second-  and 
higher-order  systems,  under  quadratic  cost  criteria,  and  find  strategies  which  are  optimal 
over  the  affine  class. 

In  Chapter  5  we  generalize  the  results  on  the  decentralized  two-person  teams  of 
Chapter  2,  by  allowing  the  action  of  one  agent  to  be  transmitted  to  the  other  agent 
through  a  number  of  noisy  channels  simultaneously,  instead  of  being  transmitted  through 
a  single  noisy  channel.  In  Chapter  6  the  results  of  Chapter  2  are  extended  in  another 
direction,  viz.,  to  the  case  of  more  than  two  decision  makers. 

In  Chapter  7  we  expand  on  the  f ramework  by  allowing  an  incomplete  statistical 
description  of  the  channel  used  to  transmit  measurements  between  the  decentralized 
agents,  and  seek  optimal  solutions  under  a  worst  case  analysis.  It  is  assumed  that  the 
unknown  pari  of  the  channel  noise  is  controlled  by  an  adversary  or  “jammer,”  and  the 
situation  is  viewed  as  a  zero  sum  game.  The  problems  are  studied  for  a  variety  of  fidelily 
criteria,  under  hard  and  soft  power  constraints,  for  the  transmitter  as  well  as  the  jammer. 


Finally,  Chapter  8  provides  a  recapitulation  of  the  results  obtained,  and  concludes 
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CHAPTER  2 


THE  DECENTRALIZED  TWO-PERSON  TEAM 


2.1.  Introduction 

In  this  chapter  we  formulate  and  analyze  some  fundamental  classes  of  stochastic 
team  problems  involving  two  decentralized  agents,  where  the  action  of  one  agent  affects 
the  information  of  the  other,  and  the  other  agent  does  not  have  access  to  the  information 
upon  which  the  first  one  acted.  In  Section  2.2  we  formulate  the  general  two-person  sto¬ 
chastic  team  problem  with  quadratic  costs  and  Gaussian  noises.  In  Section  2.3  we  identify 
those  instances  of  the  general  problem  for  which  the  optimal  solutions  are  linear.  In  Sec¬ 
tion  2.4  we  show  that  for  some  instances  in  which  we  cannot  show  that  linear  solutions 
are  globally  optimal,  the  optimal  linear  policy  may  be  outperformed  by  an  appropriate 
nonlinear  strategy.  In  Section  2.5  we  comment  on  some  aspects  of  the  difficulties  associ¬ 
ated  with  Witsenhausen’s  counterexample,  (Witsenhausen  [1968]),  and  the  concluding 
remarks  in  Section  2.6  end  this  chapter. 

2.2.  Problem  Formulation 

In  this  section,  we  formulate  a  two-person,  decentralized,  stochastic  team  problem 
with  nonclassical  information,  where  the  action  of  one  agent  affects  the  information  of  the 
other  and  the  information  structure  is  nonnested  (Basar  and  Cruz  [1982]).  The  problem  is 
to  design  the  controls  u0  and  Uj  so  as  to  minimize  the  quadratic  cost  J(y0,y, ),  where 

J(y0.yi)  =  E[k0u02  +  s0u0x  +  sut2  +  s,u,x] , 


with 


ui  =  yi(y). 


>  0,  s  >  0.  s0  and  sx  being  prespecified  constants. 

The  variables  z  and  y,  upon  which  the  two  agents  base  their  respective  actions,  are 


given  by 

z  =s  ax  +  v 

(2.1) 

and 

y  =  Uq  +  bx  +  w  . 

(2.2) 

Here  x,  v  and  w  are  zero  mean  Gaussian  random  variables,  which  are  independent  of 
one  another,  and  have  variances  oK2,  aT2  and  a  l,  respectively. 

The  situation  is  depicted  schematically  in  Figure  2.1  below. 


b 


Figure  2.1.  The  two-person,  decentralized,  stochastic  team. 

There  are  no  general  analytical  or  numerical  tools  which  can  be  used  to  obtain  the  optimal 
solution  of  this  problem,  the  difficulty  arising  mainly  due  to  its  nonclassical  information 
structure.  The  separation  principle  (Wonham  ll%8]),  for  example,  does  not  apply  here. 


since  the  estimation  part  cannot  be  separated  from  the  control  action.  Numerical  tech¬ 
niques  fail  to  provide  any  answers  because  the  discretized  version  is  NP  complete  (Tsit- 
siklis  and  Athans  [1985]).  To  further  understand  the  nature  of  the  underlying  difficulty, 
we  may  view  the  above  team  problem  as  an  equivalent  control  (one-person)  problem,  with 
the  control  exhibiting  a  dual  role.  We  complete  the  squares  to  obtain  the  cost  functional 

Sj 

E[k0u02  +  SqUqX  +  s(uj  +  —  x)z] , 

2s 

which  implies  that  the  optimal  y1  is  necessarily  given  by 


ui  =  ViCy)  - - E(xly) 

2s 


(2.3) 


and  hence  we  have  the  equivalent  problem: 


si 

Minimize  E[k0(u0  -i - —  x)z - E(E(x  I  y)z)J 


Vo 


2k 


4s 


subject  to  (2.2). 

We  thus  have  a  quadratic  term  to  be  minimized  as  is  usual  in  stochastic  control  for¬ 
mulations;  in  addition,  there  is  a  second  nonquadratic  term  which  is  influenced  by  the 
information  about  x  that  y  contains.  In  the  absence  of  this  second  term  the  minimum  is 
attained  by  choosing  u0  as  a  linear  function  of  z: 


so 

u0  =  y0(z)  - - E(x  I  z)  =  - 

2k0 


(2.4) 


but  the  presence  of  this  second  term  brings  in  the  possibility  of  a  conflict  between  the  two 
roles  of  control  and  information  transmission.  Thus,  even  if  affine  laws  are  optimal. 
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indirect  techniques  need  to  be  used  to  prove  their  optimality;  but  in  fact,  as  to  be  eluci¬ 
dated  below,  affine  strategies  do  not  always  continue  to  be  optimal. 

23.  Instances  with  Optimal  Linear  Solutions 

In  this  section  we  show  that  if  either  b  =  0  or  o  2  =  0,  then  the  stochastic  team  prob¬ 
lem  formulated  in  the  preceding  section  admits  optimal  solutions  which  are  linear  in  the 
observation  variables. 

(a)o*  =  0,  b  arbitrary 

If  ov2  =  0,  then  we  have  a  =  1  without  loss  of  generality,  and  we  define  u'0  =  u0  +  bx 
to  obtain  the  equivalent  Problem  PI  below; 

Minimize  J'Cy'o.y'j) 

y'o-y'i 

where 

2 

J'(y'o*y  V  =  Elk0u  o  +  s»  +  —  (u'i  “  x)  +  K] 

4s 


u'o  =  y'oM  • 

(2.5) 

II 

vf 

o  ' 

+ 

(2.6) 

S  Q  s:  Sq  2k0b  * 

(2.7) 

and  K  is  a  constant  independent  of  u'0  and  u'r 

We  thus  obtain  a  problem  of  the  type  studied  in  Bansal  and  Basar  [1987a],  without  a 
product  term  between  the  decision  variables,  for  which  linear  strategies  have  been  shown 
to  be  optimal.  We  briefly  outline  the  approach  here  for  completeness.  Note  that  the  situa¬ 
tion  is  as  depicted  in  Figure  2.2.  In  view  of  the  discussion  in  Section  1.3,  in  which  prob- 
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lems  with  nonclassical  information  have  been  related  to  problems  involving  information 
transmission,  the  agent  taking  action  u0  may  be  viewed  as  a  generalized  encoder  and  the 
agent  taking  action  ut  as  a  decoder,  in  the  terminology  of  the  information  transmission 
system  introduced  in  Chapter  1. 


Encoder 

_  u  A 

V 

Decoder 

1  0  . 

/u 

Figure  2.2.  The  transformed  Problem  PI. 

We  first  consider  the  transformed  problem  under  the  additional  restriction 

E[u'02]  <  P2  .  (2.8) 

Note  that 

2 

1  °x 

I(x;y)  >  I(x;u'j)  ^  —  log -  (2.9) 

2  E[(u'j-x)2] 

where  I(a;b)  denotes  the  mutual  information  of  random  variables  a  and  b.  The  first  ine¬ 
quality  in  (2.9)  is  the  data  processing  inequality,  and  the  second  inequality  follows  from 
the  definition  of  mutual  information  (see,  for  example.  Wyner  [1970]). 

Further,  we  have 


I(x;y)  =  H(y)-H(y  lx) 


(2.10) 


r^TVf.l  ' 


where  H(y)  is  the  entropv  of  the  random  variable  y  and  H(ylx)  is  the  conditional  entropy 
of  y  given  x. 

Using  (2.10)  along  with  the  fact  that  for  a  given  variance  the  entropy  is  maximized 
by  a  Gaussian  random  variable  with  that  variance,  (see,  for  example,  Kagan,  Linnik  and 


Rao  [1973]),  we  have 


r.2  ,  2 

1  P  +aw 

I(x;y)  <  —  log( - ) . 


(2.11) 


Using  (2.1 1)  along  with  (2.9)  we  obtain  (under  the  restriction  (2.8)) 


E[(u't  —  x)2]  ^ 


2  2 


P2+^w 


(2.12) 


Now  let  ]'  denote  the  minimum  of  J'Cy^.yj)  under  the  hard  power  constraint,  i.e., 


J'p  4  inf  Jty'o.y'i) . 
u'#.u',.E(u'0J]=P2 


(2.13) 


We  then  have 


J'  >  k0P2  +  Inf  E[s'0u'0x]  +  Inf  Eftu',  —  x)2]  —  +  K 


Htu'0]=PJ 


Eiu’/]=P^ 


>k0P2+  Inf  E[s'0u'0x]  +  Inf  E[(u\  -  x)  ]  —  +  K 

Elu'^P2  Etu'02I^P2  4s 

2  2  2 

2  S1 
=  k0P  -  I  s  0  I  Pox  + -  —  +  K 

(P2+o,J)  4s 

2  2  2 


(2.14) 


>  Min[k0P2  —  ISoIPa^ 
p^o 


°\  °w  S1 


—  ]  +  K 


(P2+a2)  4s 


=  k0P*  —  I  s'0  I  P*ax  + 


’2  2 
°x°w  S1 


(P^a;.) 


—  +  K 
4s 


r 
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We  thus  obtain  a  lower  bound  on  the  optimal  cost,  and  the  final  task  is  to  note  that 


this  lower  bound  is  tight  and  is  achieved  by  using  the  linear  policy: 


u*0  =  -  (sgn  s'0) —  x  . 

o„ 


We  now  have  Theorem  2.1  below: 


Theorem  2.1. 


(2.15) 


(i)  The  stochastic  team  problem  of  Section  2.2  with  a  2  =  0,  admits  an  optimal  solu¬ 


tion  which  is  linear  in  the  observation  variable and  is  given  by 


u*o  ~  y’ Vx)  =  P*x 


where 


Sj  (p*+b)o2 

u*!  =  y*,(y)  - - -  y 

2s  (p*+b)2ox+a^ 


p*  =  \*  -  b 


and  K*  is  given  by  the  solution  to  the  following  parameter  optimization  problem: 

2  2  2 

r  2  2  2  S*  i 

X*  =  arg  mintkgA  ax  +  s0 kox  + - J . 

\  4s  fi2  2  .  2-. 

U  ax  +aw) 


(ii)  The  optimal  value  of  the  cost  is 


2  2  2 

koX*2a  2  +  s>*ox2  +  - - - +  K 

4s  u*V+0 


(2.16) 


(2.17) 


(2.18) 


(2.19) 


where 


K  =  ox2(— k0b2  +  s0b  —  s“/4s)  . 


Remark  2.1.  The  parameter  optimization  problem  (2.18)  always  admits  a  solution  since  as 
\ — »oo,  so  does  the  expression  to  be  minimized,  implying  that  we  can  restrict  the  search  to  a 
compact  set  over  which  a  continuous  function  always  admits  a  minimum. 


(b)  b  =  0,  ov2  arbitrary 

If  b  =  0,  then  the  channel  noise  becomes  independent  of  the  input  x,  and  the  problem 
is  a  special  case  of  the  multipath  system  considered  in  Bansal  and  Ba$ar  [1987b]  for  which 
linear  strategies  have  been  shown  to  be  optimal.  For  completeness  we  outline  the 
approach  here.  We  can  assume,  without  any  loss  of  generality,  that  a  =  1,  then  introduce 
the  random  variable 

m  =  E(x  I  z)  (2.20) 

and  make  the  following  observations: 

si 

(i)  -y,(y)  = - E(xly) 

2s 

(ii)  E[E(x  I  y)  -  x)2]  =  E[(E(x  1  y)  -  m)2]  +  E[(m  —  x)2] 

(iii)  E[u0(z)x]  =  E[u0(z)m] 

(since  the  random  variable  (x-E(xlz))  is  independent  of  z). 

(iv)  The  random  variable  m  is  Gaussian  distributed,  since 

2 

m  = - z, 

2  ,  - 
ox+ov 

and  has  the  variance 

2  2  .  2> 

=  +  Oy). 
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In  view  of  the  above  observations,  the  problem  with  b  =  0  is  equivalent  to 


Minimize  J"(y'o.yi) 
y'o-yv 


where 


J'/(u'0,u1)  =  E[k0u'02  +  s0u'0m  4-  —  (u,  —  m)  4  K'] 

4s 


u'o  =  y '<>(») 

Uj  =yi(y) 


and  K'  a  constant  independent  of  u'0  and  ur  Since  m  is  zero-mean  Gaussian,  we  have  a 


problem  of  the  type  discussed  in  part  (a)  of  this  section,  and  the  optimality  of  linear  stra¬ 


tegies  follows. 


Theorem  2.2. 


(i)  The  stochastic  team  problem  of  Section  2.2  with  b  =  0,  a  —  1.  admits  an  optimal 


solution  which  is  linear  in  the  observation  variables,  and  is  given  by 


u*0(z)  =  X*z 


u*1Cy)  =  — — 


1 X*  (ax4o')4aw 


where  X*  is  given  by  the  solution  to  the  following  parameter  optimization  problem: 


2  2,- .  2  2  ,  2, 

2,2  2-,  2  a*U  av+°w) 

X*  =  arg  min  ICqX  (ax  4  av )  4  SgXax  4  —  - 


4s  K2/  2.„2, 


‘X  (ox4av)4ow 


(a  solution  to  which  may  always  be  found  as  discussed  in  Remark  (2.1)). 


(ii)  The  optimal  value  of  the  cost  is 

,  ,  ,  s2  a/CX^o^+o*) 

V^Ca  2  +  a2)  +  SoX*a  2  + - 

4s  (X^Co  2+cr  2)+a  2) 


2.4.  Nonoptimality  of  Linear  Strategies 

In  the  preceding  section  we  have  identified  those  instances  of  the  general  two-person, 
decentralized,  stochastic  team  problem  for  which  the  optimal  strategies  are  linear  and 
may  be  obtained  through  the  solution  of  a  related  parameter  optimization  problem.  These 
include  the  cases  where  either  ov2  =  0,  i.e.,  an  uncorrupted  version  of  the  variable  x  is 
available  to  the  agent  acting  first;  or  b  =  0,  i.e.,  the  channel  noise  is  independent  of  the 
input  x.  In  this  section  we  show  that  when  neither  of  the  above  two  conditions  hold,  then 
it  is  possible  to  construct  some  problem  instances  where  the  optimal  linear  strategy  is  out¬ 


performed  by  an  appropriately  chosen  nonlinear  strategy.  In  the  following,  we  first 
assume  that  b  =  1,  a  2  =  0.  Note  that  if  these  assumptions  imply  that  linear  strategies  are 


not  optimal,  then  we  cannot  expect  linear  strategies  to  be  optimal,  in  general,  for  the  class 
of  two-person  decentralized  teams  with  nonzero  b  and  o  2. 


Problem  PG 


where 


Minimize  E[k0u02  +  s0u0x  +  su  2  +  SjUjx] 


uo  =  y</z)  -  ui  =yj(y) 


Note  that,  since  z  =  ax  +  v,  x  may  be  written  as  x  =  0z+n  where 


o  2  2  .  2^ 

0  =  aox  /(a  o  +  ov  ) 


rv 


and  n  is  a  Gaussian  random  variable  which  is  independent  of  z,  with  variance 


2  2  2lf  2  2  .  2> 
°n  =  °x  ®r/Ca  ax  +  ov) . 


Thus  we  obtain  the  equivalent  problem 


Minimize  Elk^2  +  SsqUqZ  +  su  2  +  Sj0UjZ  +  SjUjn] 

v,.y, 


where 


u0  =  •  U1  =  Ti(u0  +  3z  +  n) . 

(In  this  restatement  we  have  used  the  fact  that  E[u0n]  =  0,  n  being  independent  of  z,  and 
having  zero-mean.)  We  next  define  u0  +  /3z  =  u'0  to  obtain  the  cost  functional 

Elk0(u'0  —  0z)2  +  0so(u'o  -  0z)z  +  su  2  +  s,0UjZ  +  s^n] 

and  arrive  at  the  problem  below: 

Minimize  E(k0u'02  +  (0so  —  2k0/3)u'0z  +  su  2  +  SjUjn  +  s,0Ujz] 

y’d-Yi 

where 


u'o  =  yVz^  •  Ui  =yl(u'0  +  n)  =  y1(y). 
We  thus  have  the  situation  depicted  in  Figure  2.3  below. 


Figure  2.3.  Schematics  for  the  transformed  problem. 

Note  the  presence  of  the  term  E(ujn)  in  the  cost  functional;  in  the  absence  of  this  term  the 
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optimality  of  linear  strategies  may  be  established  via  information  theoretic  bounds.  We 
now  have  a  problem  which  is  similar  to  the  Gaussian  test  channel  problem  (Gallager 
[1968]),  with  the  difference  that  instead  of  having  to  estimate  the  input  z,  the  decoder  is 
required  to  estimate  a  linear  combination  of  the  input  and  the  channel  noise.  We  thus 
have  the  problem  below: 


where 


Minimize  E[k0u'02  +  s'qU'qZ  +  s(ux  +  snn  +  s12z)2] 
r'o-Yi 


u'o  =  y'o(z^  •  *  VjCu'o  +  n) 

®  o  =  #(so  2kp) 
sn  =  Sj/2s 
s12  =  s13/2s  . 


We  now  note  that  since  n  =  y  —  u'0,  we  have 


ui  +snn  +  s12z 

=  ut  +  su(y  -  u'0)  +  s12z 
=  Uj+sny +  s12z-snu'0. 

Defining 


u'j  =Uj  +sny 


we  obtain  the  problem 


Minimize  E(k0u'02  +  s'0u'0z  +  s(u',  +  s,2z  —  suu'0)z] 
Yo’Y\ 


where 


u'0  =  y'</z)-  u'i 


Th.j  is  a  problem  of  the  type  studied  in  Bansai  and  Basar  [1987a]  with  a  product  term 
between  the  decision  variables,  and  hence,  as  shown  there,  regions  of  the  parameter  space 
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exist  where  nonlinear  strategies  outperform  the  optimum  linear  strategies. 


An  Illustration 

Consider  the  Problem  PG  with  parameter  values  k0  =  0.01,  s0  =  6.02,  s=1.0, 
Sl  =  2.0,  ox2  =  1.0025,  a  =  —  0.5/(1.0025)  and  ov2  =  100-C0.5)2/ 1.0025. 

With  the  above  choice  we  may  write  x  =  /3z  +  n  where  /3  =  —0.005  and  on2  =  1.0. 
(Recall  that  we  have  taken  a  2  =  0,  b  =  1  and  a  2  =  100.0). 

We  then  obtain  the  transf ormed  problem 

Minimize  E[O.01u'o2  -  0.03u'0x  +  (u'j  -  u'0  -  0.005z)2] 


where 


u'0  =  y'o 

u',  =y',(u'0  +  n)sy,i(y). 

Now,  if  y'0  is  linear,  then  optimal  y\  is  also  linear,  and  hence  restricting  y'0  to  the  form 
y'0  =  \z,  and  optimizing  over  X,  yields  a  cost  of  -1.2477  which  is  the  best  in  the  linear 
class.  However,  we  can  show  that  the  nonlinear  policy 

y'0(z)  =  10  sgn  z 

,  ,  ,  10  if  y  >  0 

y  1  y  -10  if  y  <  0 


yields  a  cost  of  -1.391 1  which  is  superior  to  the  optimal  cost  in  the  linear  class. 


2.5.  Some  Comments  on  Witsenhausen's  Counterexample 


The  following  problem  Wl,  studied  by  Witsenhausen  [1968],  is  a  two-person,  decen¬ 


tralized,  stochastic  team  problem  with  a  product  term  between  the  decision  variables,  and 


may  be  viewed  as  a  special  case  of  the  reduced  version  found  in  Section  2.4. 


Problem  WI 


Minimize  E[k0(u0  —  x)2  +  (u0  —  u^2] 

Va-Vi 


where 


u0  =  y0^);u1  =  y1(y) 


y  =  u0  +  w  . 


Noting  that 


(u0  -  Uj)2  =  (y  -  w  -  Uj  )2 


and  defining 


u  -  =  y  -  u, 


we  obtain  the  equivalent  Problem  WI'  below. 


Problem  WI' 


Minimize  E[k0(u0  —  x)2  +  (u'j  —  w)2] 
yo-y'i 


P 


ft 


I 

* 


Jl 

it 
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Uq  =  y0(x)  ;  u'j  =  yVuo  +  w)  . 

We,  therefore,  have  a  problem  In  which  the  second  agent  wishes  to  estimate  the  channel 
noise  (which  is  independent  of  u0),  in  sharp  contrast  to  the  communication  type  problems 
where  the  second  agent  wishes  to  estimate  the  observation  of  the  first  agent. 

Since  the  second  agent  necessarily  uses  u'j  =  E(wly),  the  second  term  in  the  cost 
function  for  Problem  WI'  becomes 

J2(y0)  =  E((w  -  E(w  I  y))2] . 

It  is  a  known  result  in  probability  and  statistics  that  for  every  fixed  power  level 
E[u02]  =  P2,  the  linear  strategy 

uo  =  VoW  =  —  x 

ax 

maximizes  J2(y0),  when  y  =  u0  +  w.  To  prove  this  result,  consider  a  zero  sum  game  in 
normal  form  with  kernel 

G(u0,Uj)  =  E[(w  -  Uj)2] 

which  is  to  be  maximized  by  a  choice  of  u0  =  y0(x)  and  minimized  by  a  choice  of 
Uj  =  y^y),  subject  to  the  constraint  E[u02]  =  P2.  Note  that 

Max  J2(u0)  =  Max  Min  G(u0,u,)  , 

U0  u0  U1 

i.e.,  the  unrestricted  maximum  of  the  function  J,  is  equal  to  the  lower  value  of  the  game 
with  kernel  G.  But  the  upper  and  lower  value  of  a  game  are  equal  if  a  saddle  point  exists, 
and  to  complete  the  proof  one  can  show  that 
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u0  ~  N(0,P2) 


I t+ol 


provide  a  saddle-point  solution  for  the  game  above. 


In  view  of  the  above  result,  we  see  that  for  a  given  power  level  the  linear  choice  of 


strategy  y0  is  the  worst  possible  design  as  far  as  the  minimization  of  the  second  term  in  the 


cost  functional  for  Problem  WI'  is  concerned.  It  is  therefore  not  surprising  that  linear 


solutions  are  not  optimal  for  the  classes  of  problems  which  involve  the  estimation  of 


channel  noise  as  in  the  case  of  Witsenhausen’s  counterexample,  as  well  as  in  the  general 


case  of  Section  2A. 


2.6.  Conclusion 


In  this  chapter  we  have  formulated  and  analyzed  some  fundamental  classes  of  sto¬ 


chastic  team  problems,  involving  two  decentralized  agents,  where  the  action  of  one  agent 


affects  the  information  of  the  other,  and  the  information  structure  is  nonnested.  We  have 


identified  those  instances  of  the  general  problem  for  which  the  optimal  solutions  are 


linear.  For  some  instances  in  which  we  cannot  show  that  linear  strategies  are  globally 


optimal,  we  have  shown  that  the  optimal  linear  policies  may  be  outperformed  by 


appropriately  chosen  nonlinear  strategies.  We  have  commented  on  some  aspects  of  the 


difficulty  associated  with  Witsenhausen's  problem.  The  important  conclusion  that  can  be 


drawn  from  the  analysis  in  this  chapter  is  that  for  two-person  decentralized  stochastic 


team  problems  there  do  exist  classes  of  parameter  values  for  which  the  optimal  solutions 


are  linear  and  may  readily  be  found,  despite  the  fact  that  the  general  problem  is  quite 


intractable. 
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CHAPTER  3 

SIMULTANEOUS  COMMUNICATION  AND  CONTROL: 

FIRST-ORDER  ARMA  MODELS  WITH  FEEDBACK 

3.1.  Introduction 

In  this  chapter  we  consider  stochastic  dynamic  teams  where  at  each  step  two  con¬ 
secutive  decisions  must  be  taken,  one  being  what  information-bearing  signal  to  transmit 
and  the  other  regarding  what  control  action  to  exert.  The  organization  of  this  chapter  is  as 
follows.  In  Section  3.2  we  shall  formally  pose  the  problem  with  hard  power  constraints 
as  Problem  PI.  In  Section  3.3  we  shall  use  an  intermediate  Problem  P2  to  construct  Prob¬ 
lem  P3.  Problems  PI  and  P3  will  be  shown  to  be  equivalent  in  the  sense  that  the  optimal 
solution  of  one  may  be  constructed  from  the  optimal  solution  of  the  other.  In  Section  3.4 
we  formulate  and  solve  an  auxiliary  problem  using  some  results  from  Information 
Theory.  In  Section  3.5  we  shall  show  that  the  solution  to  the  auxiliary  problem  may  be 
used  to  provide  a  solution  to  Problem  P3,  thereby  solving  Problem  PI.  In  Section  3.6  we 
study  the  associated  "soft”  constraint  version,  and  in  Section  3.7  we  study  the  existence 
of  stationary  optimal  policies  for  the  infinite  horizon  problem  with  discounted  cost.  The 
concluding  remarks  in  Section  3.8  then  end  this  chapter. 

3.2.  Problem  Formulation 

In  this  section  we  give  a  precise  formulation  of  the  problem  to  be  solved  in  the 
sequel. 

The  problem  is  to  control  a  stochastic  dynamic  system,  the  description  of  which  is 
available  in  the  form  of  a  general  first-order  difference  equation,  with  the  objective  being 
to  minimize  a  quadratic  cost  functional. 


v  wvwvn  *  V  x  a  w » v  rt  n  i  ct  \  « n»  grp  g*  g*  vn  nn  wi  w  k*  xtxt  wunw 
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The  control  v;  is  based  on  the  observation  vector  (y0,..,yj),  where  each  of  the  yk’s  is  a 
noise-corrupted  version  of  the  information-bearing  signal  U;,  the  Uj’s  being  based  on  the 
current  value  of  the  state  and  all  previous  yk’s.  In  the  context  of  the  space  probe  example 
of  Section  1.1,  (with  the  u;’s  being  generated  on  board  the  space  probe  and  the  v;’s  being 
generated  on  the  earth  station),  such  a  situation  arises  quite  naturally  because  a  feedback 
line  (the  uplink)  is  available  which  is  many  times  more  reliable  than  the  forward  link 
(the  downlink). 

The  problem  clearly  has  a  nonclassical  information  structure,  since  the  agent  taking 
action  V;  does  not  have  access  to  the  information  based  on  which  action  u;  is  taken.  A  pre¬ 
cise  formulation  of  this  nonclassical  stochastic  team  problem  is  provided  next. 

We  specify  the  stochastic  system  by  the  following  set  of  equations: 

xi+,  ~  Pixi  +  mj  -  v,  (3.1a) 

y;  =  Uj  +  Wj  (3.1b) 

along  with 

Eluj2]  <  Pj2  (3.1c) 

Uj  =  hjUj.y1-1)  (3.1d) 

and 

v,  =  Yi(y‘)  •  ( 3. 1  e ) 

Here  (3.1a)  and  (3.1b)  are  the  state  and  measurement  equations,  respectively.  The  sub¬ 
script  i,( i=0, 1 .... )  denotes  that  the  realization  of  the  random  variable  is  at  the  i-th  time 
instant,  whereas  the  superscript  denotes  the  history  of  the  random  variable  up  to  that 


time  instant,  i.e., 


which  we  are  also  going  to  interpret  as  a  row  vector.  The  random  variables  x0,  w;  and  rrij 
(i>0)  are  all  assumed  to  be  independent  and  Gaussian  with  mean  zero  and  variance  a(2)t 

the  subscript  being  the  identifier.  The  functions  hs  and  y;,  i-0,1 . are  the  control  policies, 

each  one  Borel-measurable  in  its  arguments,  and  leading  to  second-order  random  variables 
Uj  and  vj(  respectively,  the  former  also  satisfying  the  power  constraint  (3.1c). 

The  criterion  for  comparing  different  control  policies  hj’s  and  y^s  is  based  on  the  cost 
function  we  wish  to  optimize,  which  in  this  case  is  taken  to  be 

N 

J(hN,yN)  =  E[  £  (ai+1x2j  +  bsv2)]  (3.2) 

i=0 

where  ai+1>0  and  b;>0  for  all  i-0,...,N,  and  the  action  variables  u>  and  v;  are  related  to 
the  policy  variables  h;  and  y i  via  (3.1d)-(3.1e).  We  thus  have  Problem  PI  below. 

Problem  PI: 

Minimize  J(hN,yN) 

hN,yN 

subject  to  (3.1a)  through  (3.1e),  where  J(hN,yN)  is  defined  by  (3.2). 

3^3.  Construction  of  an  Equivalent  Problem 

The  gist  of  this  section  is  as  follows:  First  an  equivalent  Problem  P2  is  constructed 
from  PI  which  differs  only  in  the  form  of  the  cost  function.  The  cost  function  for  P2  is 
in  the  form  of  a  sum  of  the  squar:.'  differences  between  state  and  control  variables.  In 
the  transformation  from  PI  to  P2  the  constraints  represented  by  (3.1a)  through  (3.1e)  are 
unaltered.  In  a  follow-up  step  Problem  P3  is  constructed  from  Problem  P2  such  that  the 
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structure  of  the  cost  function  is  left  unaltered,  while  the  state  equations  are  redefined  so 
as  to  facilitate  subsequent  analysis. 

These  two  transformations  are  presented  below  as  Claims  3.1  and  3.2,  respectively. 


Claim  3.1:  Under  the  set  of  constraints  represented  by  (3.1a)  through  (3.1e),  the  cost 
function  for  Problem  PI,  defined  by  (3.2),  is  identical  to 


J"(hN,yN)  =  E[£  a'^Vj-b'jXj)2]  +  cN 


where 


b'i4kl+iPi/(bi+ki+1) 


a';  4  bj+ki+1 


cN4k0ox2  +  Z  kl+1a*( 


(3.4a) 

(3.4b) 


(3.4c) 


and  fkj  is  a  sequence  defined  recursively  by 


ki  =  ai  +  ^i+ibiPi  /(bj+ki+l) 


kN+l —  aN+l 


Proof:  This  is  a  standard  result  in  stochastic  LQ  control  with  perfect  state  information 
(see,  e.g..  Bertsekas  [1987]  or  Kumar  an  i  Varaiya  [1986]),  known  also  as  "completing  the 
squares."  Note  that  is  a  constant  (independent  of  the  control  sequence  {v,}),  and  (3.5) 
is  the  so-called  discrete-time  Riccati  equation  for  this  scalar  problem. 


Claim  3.2:  The  solution  to  Problem  P2  may  be  obtained  by  solving  the  following 
equivalent  Problem  P3. 
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Problem  P3: 


where 


and  the  n{s  satisf  y 


Minimize  J(hN,yN) 


xi+i  =  Pixi  +  mi 
Vi  =  YiCy1) 

Yi  =  U)  +  W; 

us»hs(xi,y,~1) 


E[Uj2]  ^  P;2 


J(hN,/)  =  E[£  a'jCvj— b'jXj)2]  +  cN  . 


(3.6a) 

(3.6b) 

(3.6c) 

(3.6d) 


(3.6e) 


Proof:  The  situation  is  depicted  in  Figure  3.1.  Substituting  for  x,  using  Equation  (3.1a) 
we  get 


Similarly. 


vi— b  ixi  =  vl+b'lvo  ~  b'1(Poxo+mo) 


v2  b2X2  =  v2+b'2Vl  +  b'2PlVo“b'2^1(PoXo+mo)+ml)- 


and  at  the  i-th  stage  we  have 


v rb'ixi  =  vi+b'ivi_1+b'ipi_,vi_2+  •  •  •  +b',  Pj_! . P^0 

-  b',(pi_1(pi_2(....(p0x0+m0)...)+mi_2)+mj_1)  . 


(3.10) 


We  now  define 


Xo  =  Xo 


X,-H=PiXi+mi  •  1=0- 1  • 


(3.11) 


»*  *«»  * i ™  * 


P.v-1 


p.v-2  *(*;  4 

i  m. 


yp  yy-t 

. — ^ 

•*4v  — 1  “ ^  ux~\  ^ 

y o  yy-2  w.v-i 


y  o  yy  -i 


y  o  yy-2 


J 

y0 

i  ■ 

r/Ts  J 

y0 

1 

1  v  1 

Figure  3.1.  Diagrammatic  representation  of  Problem  P3. 
*i  =  vi+b'lvi-1+b,lpi_1v1_2+...+b'jpi_1 . p,v0  . 

Using  these  new  variables,  the  cost  function  (3.3)  can  be  rewritten  as 

N 

J(hN,yN)  =  El£  a'^v.-b'iXj)2]  +  cN  . 

i«0 

Vj  =  ViCyV-1)  . 


(3.12) 


(3.13) 


where  the  evolution  of  the  Xj’s  is  determined  by  (3.1 1 ).  Since  this  is  a  team  problem,  and 
for  each  hxed  h  the  resulting  stochastic  control  problem  has  classical  information, 
minimization  of  (3.13)  over  (h  ,y  )  is  equivalent  to  its  minimization  over  (h  ,y  )  where 
y(  has  only  y‘  as  its  argument. 

The  relationship  between  Vj’s  and  v^s  is  given  by 


-N  NnT 

v  =  v  B 


(3.14) 


where  B1  is  the  transpose  of  the  following  nonsinguiar  lower  triangular  matrix: 


I 


a 


b'i 
b'2Pi 
b  3P2P1 


u  0  0-00 

1  0  0-00 

b'2  10-00 

b'3p2  b'3  1  •  0  0 

•  1  0 


b  nPn-i-Pi  b  nPn-1"P2  '  ^  n  * 


We  may  thus  write 


N  ~Nr_.T-.-l 

v  =  v  [B  ] 


and  we  thus  see  that  in  order  to  obtain  the  optimal  solution  to  PI,  we  may  equivalently 
solve  the  problem  in  terms  of  vs  and  x;  (over  hN,yN),  which  is  precisely  Problem  P3.  This 
then  leads  to  the  following  equivalence  between  the  solutions  of  Problems  PI  and  P3: 


Lemma  3.1. 


b 


(i)  Problem  PI  admits  a  solution  if,  and  only  if,  Problem  P3  does. 

(ii)  If  (hN,yN)  is  a  solution  for  PI,  then  (hN,yNBT)  solves  P3;  conversely,  if  (hN,yN) 
solves  P3,  then  (hN,yN[BT]  ^  is  a  solution  for  PI. 


3.4.  An  Auxiliary  Problem 

In  this  section  we  formulate  and  solve  an  auxiliary  problem  which  will  play  an 
important  role  in  the  solution  to  Problem  P3. 


J: 

i, 

1 

I. 

•i 

* 

•t 

s* 

S' 


Consider  the  situation  depicted  in  Figure  3.2,  the  problem  being  one  of  finding  the 


signals  u0 . uN  subject  to  the  power  constraints 

E[u“]  <  P,2  , 


so  as  to  minimise  the  mean-square  error  in  the  estimation  of  zN  using  y0,y, . yN.  It  is 


A  u 
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given  that  z0,  n;,  Wj,  i— 0, 1 are  mutually  independent  Gaussian  random  variables,  each 

with  mean  zero  and  variance  indicated  by  a(2 ,  the  subscript  being  the  identifier.  Follow¬ 
ing  the  convention  of  Information  Theory,  we  let  I(zN;yN)  denote  the  mutual  information 
between  zN  and  yN,  and  call  the  supremum  of  this  quantity  the  capacity  of  the 
corresponding  system. 

We  shall  first  solve  this  problem  for  the  case  when  N-l.  The  proof  for  arbitrary 
finite  N  will  then  be  shown  to  follow  a  similar  line  of  reasoning.  The  case  with  N-l  is 
illustrated  in  Figure  3.3. 

Recall  from  the  discussion  in  Section  1.3  that  for  any  channel  the  minimum  possible 
distortion  D*  that  can  result  from  its  use  is  related  to  its  capacity  by 

R(D*)  =  C  . 

First  consider  the  lower  branch  of  Figure  3.3,  redrawn  as  Figure  3.4.  We  find  D*  for 
this  system.  Let  us  suppose  that  the  input  is  connected  to  a  Gaussian  memoryless  source 
with  variance  a,2,  in  which  case  we  have 

zi 

2 

1 

R(D)  =  Max(0,—  log( - ))  .  C3.15) 

2  D 

Let  z0J  denote  the  sequence  (z0l, ....  zoj),  which  we  again  interpret  as  a  row  vector;  define 
nj,  VjJ  and  z,  likewise.  We  introduce 

( l+a,)o22 

ml  4  Hz’  I  4)  = - —I—  zj 

(  1+cx.  )'o  +0.“ 

and  observe  the  following. 


(a)  The  components  of  jn,1  are  i.i.d.  Gaussian  variates  with  zero  mean  and  variance 


?■ 


3.2.  Diagrammatic  representation  of  the  auxiliary  problem 


I 


we  have 


r*00  j 

(l+a1)2o^+o„2i  CU+a.go'+o-2)  (P„2+<dt) 

R(D*)  may  now  be  computed,  and  we  have  the  following  result: 

i 

I 

i 

|  Lemma  3.2 :  For  the  system  depicted  in  Figure  3.4,  the  mutual  information 

|  bounded  above  by 


C^log 


(P^+o^XCl+ai^a^+o^) 


Now,  consider  the  system  depicted  in  Figure  3.3.  We  first  note  that 

Kzi;y0>yi)  =  Kzf>y0)  +  K^yi  i  y0)  • 

Now, 


KZpy,  I  y0)  =  I(Zj,w;yj  I  y0)— Kwjy,  I  Zl,y0) 
<  I(Zj,w;yj  I  y0) 

=  l(w;yj  ly0) 


=  H(yj  ly0)-H(y,  I  w,y0) 
=  H(yt  i  y0)— H(y,  I  w) 
<  H(yj)-H(yj  I  w) 

=  I(yt;w) 


1 


n  2  2 

P1  +aw, 


^  —  log(- 
2 


(3.20) 


Kzi;y0) is 

(3.21) 

□ 

(3.22) 

(3.23a) 

(3.23b) 

(3.23c) 

(3.23d) 

(3.23e) 

(3.23f) 

(3.23g) 

(3.23h) 


where  HO)  is  the  entropy  and  HO  I  •)  the  conditional  entropy.  Here  steps  (a),  (d)  and  (g) 


*■>  *■  r 


follow  from  the  definition  of  mutual  information,  steps  (c)  and  (e)  are  due  to  the  Markov 
Property,  step  (b)  follows  because  information  is  always  positive,  step  (f)  is  valid  because 
conditioning  cannot  increase  entropy,  and  the  last  step  holds  because,  for  a  fixed  variance, 
the  Gaussian  random  variable  has  the  maximum  entropy  (Kagan  et  al.  [1973]). 

Using  (3.21)  and  (3.23),  along  with  (3.22),  we  obtain  the  next  result. 


Lemma  33:  For  the  system  depicted  in  Figure  3.3, 


1  (?o+alMl+a02al+anl)  Pl2+< 

i[2i;y0>y^  <  -  lQg - 

2  (<(Po2+<H(1  ait 


(3.24) 


Using  this  upper  bound  on  Kzj.y^yj)  we  can  find  a  lower  bound  on  the  minimum 
mean-square  error  achievable  when  the  problem  is  to  estimate  zx  from  the  observations  y0 


and  yr 


We  have 


I(z1;y0,y1)  ^  Kzf.Vj)  >  —  log - 

2  EKzj-Vj)2] 

which  implies  (using  Equation  (3.24)) 

,  (P„2+alX(l+«,)2c>2+a2)  P  ‘+alt  ,  a,2 

—  log - • - ^  —  log - 

2  (an2(;P02+Ow  )  aw,  2  EttZj-Vj)2] 


(3.25) 


(3.26) 


‘1' 


EKzj-v,)2]  > 


°  w,(a  P*+> a  2o )+( 1 +«J  ft O  o  l) 
(P02+a  2  XP,2+o  *  )o z2 

O  10 


(3.27) 


K«!W 


I  .  I  . 


We  next  note  that  if  we  use  the  policy 


uo  =  KZ0 


ui  =  ^•i(z1-E(z1ly0)) 


(3.28) 


with  X0  and  Xj  chosen  so  as  to  satisfy  the  power  constraints,  the  minimum  mean-square 
error  is  indeed  achieved  and  we  have 


Lemma  3.4.  The  policies  given  by  (3.28)  are  the  policies  which  minimize  the  mean-square 
error  in  estimating  zx  from  the  pair  (y^yj). 

We  now  consider  the  case  with  arbitrary  N,  depicted  in  Figure  3.2.  First,  consider 
the  problem  in  the  absence  of  the  most  recent  observation,  i.e.,  the  uppermost  channel  of 
Figure  3.2  removed.  Assuming  that  the  version  of  the  problem  with  (N-l)  channels  is 
already  solved,  the  capacity  CN_t  (i.e.,  maximum  mutual  information  between  input  and 
output)  for  the  portion  of  the  system  within  the  rectangular  box  CDEF  is  known.  We  can 
therefore  find  the  minimum  achievable  distortion  for  a  memory  less  Gaussian  source  with 
variance  when  only  the  portion  within  the  box  ABCD  is  in  use,  by  computing  the  con¬ 
ditional  estimate  of  zN  given  zN_j,  and  transmitting  this  optimally.  Using  this  minimum 
achievable  distortion  we  can  find  an  upper  bound  for  I(zN;yN_1)  as  follows  (where  is 
the  sequence  zN . zN ) 


D*  =  Lim  -  ElIz^-VjJH2 

j— oo  j 


alan  (l+crN)2o4 

*N  nN  +  N  *N  — 2Cn_, 

(l+aN)2o£+an2N  (1+afN)2°^+0nN 


(3.29) 


Lemma  JJ.  The  mutual  information  l(zN;yN)  is  bounded  above  by  CN,  which  is  the  last 
step  of  the  recursion: 


C=  —  log(- 


n  2  2 
Po+Ow„ 


•) 


(3.34a) 
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i-i  4  i  *’I 


and  for  i-1 . N 


(l+“i  )2<J?+o* 


n  2  2 
Pi  +  °w, 


q  =  -  log - • - 

2  „  2 ,  <• ,  .  i2  2  ~2Ct-i  n  2 

o.  +(l+QfJ  a,  e  aw. 


(3.34b) 


Let  Aj  denote  the  minimum  achievable  mean-square  error  when  z,  is  estimated  using  y1, 


i.e.,  (using  (3.25)) 


We  therefore  have 


A  a  2  ~2C< 
A:  4  a,  e 


D2  ,  2 
Pa  +a... 


(3.35) 


(3.36a) 


f 


3 

i 


and  for  i-l,...,N 


(  (!+<*, )20„4 


Pi2+< 


’  2 
2  % 


(3.36b) 


* 


We  shall  next  show  that  this  lower  bound  is  tight  and  may  be  achieved  byf  using  the  poli- 


Uo  =  Vo 


(3.37a) 


and  for  i-1 . N 


u(  =  A.j(Zj  Efzjy1  *)) 


(3.37b) 


Here  the  X;’s  are  chosen  so  as  to  meet  the  power  constraints  with  equality.  Since 


Jb* 

| 

s 
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■»<?*•  *»*  i*'  #*»  *>»  txi 


Zi_!  =  Cl+cr^Zj  +  nj 


(3.38) 


we  may  equivalently  write 


Zj  =  Pi_,zi_1  +  mj_, 


(3.39) 


where  p j_t  is  a  constant. 


Pi_!  4  (1+a.)- 


(3.40) 


and  is  a  Gaussian  random  variable  which  is  independent  of  zi—1  and  has  variance 


2  1  2 

°®|.1  2  %  ’ 
a „ 


(3.41) 


With  the  policies  chosen  as  in  (3.37),  we  have 


y0  =  Vo  +  wo 


(3.42a) 


and  for  i-l,...,N 


y,  *  XjCz,— H(zjy*  1 ))  +  Wj . 


(3.42b) 


Let  Ij  denote  the  mean-square  error  in  the  estimation  of  z(  from  y‘  when  the  communica¬ 


tion  strategies  are  chosen  as  in  (3.37),  i.e.. 


r,  =  E[(z-E(zi|yi))2] 


We  then  have,  using  (3.36) 


(3.43a) 
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is 
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and  for  i-1 . N 


4 


(3.43b) 


(1+ap  — — 1^1+— : 7"ant 


Further,  by  our  specific  choice  of  policy.  (Zj-ECZjly1-1))  is  a  zero  mean  Gaussian  random 
variable,  and  therefore 


EKz-ECzJy1-1)— EKz-ECzJy1  1))|yil)2]  = 


2  2 
Si  °w, 


r>2  .  2 

Pi  +  <V 


(3.44) 


where 


s2  4  E[(zj— E(Zj|yi-1))2] 

4 

oz 

=  (l+<xf—~  Vl  + 


2 

T 


(3.45) 


Also,  E(E(zi|yi-1)|yi)=0  by  our  choice  of  policy,  since  y;  is  independent  of  y1  ,  and  there¬ 
fore  the  expression  on  the  left-hand  side  of  (3.44)  becomes 

E[(Zi— E(zijy,_1 )— E(zi|yi))2] 

=  E[(  Zj — E(  zjy 1 )  )2] 


and  we  get  the  recursion  (for  i-1 . N) 


a2  (1+«,)V 


n2,  2 

P; 


(3.46a) 
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with  the  initial  condition 


2  _  2 


s.= 


P02+a^ 


(3.46b) 


The  recursion  for  the  E^s  is  therefore  identical  to  the  recursion  for  the  A’s  (given  by 
(3.36)),  which  denote  the  minimum  mean-square  error  achievable.  This  shows  that  the 
lower  bound  on  error  is  indeed  tight  and  thus  leads  to  the  following  theorem. 


Theorem  3.1. 


(a)  The  policies  given  by  (3.37)  minimize  the  mean-square  error  incurred  in  estimat- 
ing  zN  from  y  for  the  system  depicted  in  Figure  3.2,  where  the  Xj’s  are  defined  by  (3.43) 
using  the  E^s  defined  by  (3.46). 

(b)  The  minimum  mean-square  error  is  given  by  the  last  step  of  the  recursion  (3.46) 
or  equivalently  by  the  last  step  of  the  recursion  (3.36). 

□ 


3.5.  Solutions  to  Problems  P3  and  PI 


We  now  return  to  Problem  P3  defined  in  Section  3.3,  where  the  policies 


u i  =  h^x.-y'  ') 


and 


v,  =  Yi(y‘) 


are  to  be  chosen  in  order  to  minimize 


J  =  E(£  a'j( vj — b^Xj)2] 


(3.47) 


3f 
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I 


r. 


under  the  constraints  depicted  in  Figure  3.1.  . 


We  first  consider  the  minimization  of  the  N-th  term  in  the  expression  for  J,  which  is 

E[a  n(vn— b  NxN)  ]  , 


the  optimization  problem  being  equivalent  to  minimizing 

E[a  Nb  N(v  N— xN)  ]  , 


where 


'N 


VN  = 


b' 


N 


i.e.,  the  problem  is  one  of  forming  the  best  estimate  of  xN  under  the  mean-square  distor¬ 
tion  criterion.  We  now  show  that  the  situations  depicted  in  Figures  3.1  and  3.2  are  identi¬ 
cal  except  for  nomenclature.  To  show  this  equivalence  we  note  that  for  i=l . N 


Zj_j  =  (l+a^Zj  +  n; 


(3.48) 


which  implies 


zi  =  O+ttj)  —  Zj-,  +  m,_, 

o. 


(3.49) 


where  the  m/s  are  zero  mean  Gaussian  random  variables  each  with  variance 


2  2 
cj  - -  o„ 

mi  2  ni*i 


(3.50) 


-I 


We  therefore  have 


Pi_i  =  ( 1+ttj) - for  i=l,..,N 

o  2 
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and  by  defining  a.2  =  a,2  we  can  complete  the  correspondence  between  the  variables  x,’s 

x«  zo 


and  z/s  for  i-0,l,...,N. 


The  solution  to  the  problem  of  minimizing  the  mean-square  error  in  estimating  xN 


from  y0, . . . ,  yN  may  therefore  be  obtained  as  in  Section  3.4.  Using  the  techniques  of  Sec¬ 


tion  3.4  we  get 


(a)  The  minimum  mean-square  error  in  estimating  x;  using  y1  is  given  by  A;,  where 


Aj’s  satisfy  the  recursion  (for  i-1 . N) 


a,— 7-Lr(oiiVi+<1) 

P.2+o2 


with  the  initial  condition 


2  2 


_  2  ,  2 
Po  +<V 


(b)  The  optimal  encoding  strategies  are 


u0  =  h*0(x0)  =  \0x0 


and  for  i-1 . N 


u(  =  h^x'.y1  ‘)  =  A.jCXj-ECxJy'  1 )) 


where  the  A^’s  satisfy  the  recursion  (for  i=l . N) 


2  A  ,  2 

Pi-l^i-1  +  am, 


(3.51a) 


(3.51b) 


(3.52a) 


(3.52b) 


(3.53a) 
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with  the  initial  condition 


(3.53b) 


(the  Aj’s  being  as  defined  by  (3.51)). 

(c)  The  optimal  choice  for  the  y^s  is 

Vj  =  b'jHxJy') 

where  Efxjy1)  satisfy  the  recursion  (for  i-1 . N) 


(3.54) 


E(xJy')  =  pi.lEC51.l|y'  *)  +  ■ 

P,+oi 


r  2  .  ,  2 


(3.55a) 


with  the  initial  condition 


E(x0|y°)  = 


P.°*.yo 


n2,  2 

P„  +a. 

0  x„ 


(3.55b) 


(the  Aj’s  being  as  defined  in  (3.51)). 

We  finally  note  that  the  policies  which  minimize  the  mean-square  error  in  the  esti¬ 
mation  of  Xj  given  y1  for  i-0,  N-l,  are  identical  to  the  corresponding  policies  used  in  the 
estimation  of  xN  given  yN,  and  we  therefore  have  the  following  theorem. 


Theorem  3,2. 

(a)  The  optimum  policies  h(  and  for  Problem  P3  are  given  by  (3.52)  and  (3.54), 
respectively,  using  the  A.j’s  and  A/s  as  defined  by  (3.53)  and  (3.51),  respectively. 

(b)  The  minimum  value  of  the  cost  function  for  Problem  P3  is 
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is 
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J«=  Ia',b'i2A,  +  cN. 

i-0 


Now  we  turn  to  the  original  Problem  PI  formulated  in  Section  3.2.  Taking  the  difference 


3 


i— 1 


i-1 


and  using  (3.54),  (3.55)  and  (3.12),  we  find  that 


vi  .  ...  ’H  Pi 


—  =  CPi-.-b',-,)  (pi:,Al-,+o2  fy, 

bi  bi-l  P:+0_. 


which  implies  that  the  optimal  control  policies  for  the  original  problem  are 

v*i  =  y*i(y')  *  b'j  Etxjy1)  4  b'jXj 

where  xf  4  E(xj|y1)  satisfy  the  recursion  (for  i-1 . N) 

P; 


X;  =  (pi_1-b'i_1)xi_,  +  — —  +  )Vlyi 


n2  .  2 
Pj  +aw 


with  the  initial  condition 


P0<V 


xo  = 


n  2  2 
Po+°x 


y0  - 


the  A/s  being  as  defined  in  (3.51). 


We  therefore  have  the  following  theorem. 


(3.56) 


(3.57) 


(3.58) 


(3.59a) 


(3.59b) 
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i 

;»* 


Theorem  33. 


(a)  The  optimum  policies  {h*}  and  {y*}  for  Problem  PI  are  given  by  (3.52)  and 
(3.58),  respectively,  using  the  A/s  and  A;’s  as  defined  by  (3.53)  and  (3.51),  respectively. 

(b)  The  minimum  value  of  the  cost  function  for  Problem  PI  is 


J*=Ia’,b-i2A,  +  cN. 


An  Illustration 


Consider  the  case  with  N-2,  the  stochastic  system  model  being  given  as 


Xj  =  x0  +  m0  -  v0 
x2  =  Xj  +  mj  -  Vj 
x3  =  x2  +  m2  -  v2  , 


and  the  objective  being  to  minimize 


J(hV)=I(x;2+1  +  vt2), 


subject  to  the  constraints  of  Equations  (3.1b)  through  (3.1e),  it  being  given  that 

2  2  2  „  2  ,  _ 

Ox0=Ow,  =  Om,  =  Pi  =  mo¬ 


using  Claim  3.1,  (Section  3.2),  we  get  the  equivalent  cost  functional 


I  a'i(v,  ~  b'iX;)"  +  c2 


3f  i 


r  w.  W 


where 
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a'0  =  13/5  ,  a',  =  5/2  ,  a'2  =  2  , 
b'0  =  8/13  ,  b'j  =  3/5  ,  b'2  =  1/2  , 
and 

c2  =  613/130  . 

Using  Theorem  3.3  above,  we  obtain  the  optimal  strategies  for  the  problem: 

u*o  *  Mo 

u*1  =  XjCxj— E(xx  I  y0)) 

^*2  =  MX2-E(x2|y1)) 


where 


where 


Xq  =  1  ,  X^  =  2/3  and  X22  =  4/7  , 


v*0  =  8/3  x0 
v*j  =  3/5  Xj 
v*2=1/2x2 


x0=  1/2  y0 

xx  =  5/26  y0  +  3/34  yx 

x2  =  1/3  y0  +  3/2  yx  +  7/ 16  y2 


E(Xj  I  y°)  =  7/26  y0 
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E(x2  ly  )  =  1/13  y0  -f  3/10  y, 


Further,  the  optimal  cost  is  6.1852. 
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&6.  The  Soft  Constraint  Version 

We  consider  again  the  problem  depicted  in  Figure  3.1,  but  now  with  the  power  con¬ 
straints 


E{Uj2]  <  P2 

removed,  i.e.,  the  optimum  power  levels  are  also  to  be  determined  via  the  underlying 
optimization  problem.  Such  a  formulation  is  useful  in  situations  where  more  "costly" 
measurements  that  contain  more  useful  or  reliable  information  may  be  used.  It  may  be 
possible  to  transmit  a  larger  power  at  the  encoder  Cat  additional  cost)  in  order  to  further 
decrease  the  mean-square  error  at  the  decoder,  this  tradeoff  being  reflected  by  the  cost  cri¬ 
terion.  Mathematically,  we  may  represent  this  as  a  power  constraint  which  is  "implied"  or 


“soft",  appearing  as  an  additional  term  in  the  cost  functional,  which  now  becomes 


JChN,/)  =  E[£  (a'i(vrb'ixi)2  +  qiu2)] . 


(3.60) 


We  shall  obtain  the  solution  to  the  soft  constraint  version  by  using  the  solution  to  the 
hard  constraint  Problem  PI  found  in  Section  3.5.  Let  Jp  denote  the  infimum  of  J  under 


the  hard  power  constraints, 


J  A  Inf  J(hN,yN) 

hN;yN;E[h,2]=P12,fif 


We  then  have  the  following  series  of  equalities  and  inequalities: 


(3.61) 


S.  tjt  *.•  «.*  | 


JP  =  Z  q,P,!  *  w  U  (*>-»>/> 

i«o  Sh^P,2  i=o 

N  N  | 

>  £  q^2  +  Inf  E  £  Ca'iCv.-b'jX;)2) 

i*0  Hlh,2]  ^  P,2  j-*g  | 


y 

y 
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S 
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=  Z  1??  +  Z  »'Pi% 


^  Min  ^  qjP,2  +  a'jb'^Aj 
P.^0  j=o  I 

N 

=  IqiP*2  +  aVi2A*i 


(3.62) 


where  the  Aj’s  are  defined  in  (3.51)  and  A*’s  are  defined  recursively  likewise,  with  Pi 
replaced  by  P*;,  i.e.,  for  i-l,...,N 


(Pi-.iV.  +  < , ) 

P*,+Pw. 


(3.63a) 


with  the  initial  condition 


A*o  = 


2  2 
°v°x0 


n*2  ■  2 

P*o  +°w. 


(3.63b) 


In  order  to  find  the  optimal  power  levels  (P*2’s),  we  can  solve  the  following  problem 


Min  £  3ipi2  +  a'jb'^A; 

P» . PN  i=0 


(3.64) 


which  is  a  nonlinear  optimal  control  problem,  the  solution  to  which  is  given  by  the  fol¬ 
lowing  dynamic  program  (with  p_j  4  1  and  a  2  ^  4  0),  where  W^A)  is  the  "optimum  cost 

to  go"  given  that  the  system  is  at  state  A  at  stage  i 


$ 

1 

s 


»* 

i 

•» 


e 

s 

V 


I 


$ 

1 

I® 


71 

I 

v.!i 


$ 

>!* 

I 


I 

I 


1 


9 


6 


'S 

’ll 

5! 


§ 

>*? 

a 

a 

1 

ii 


WK+1=0 


Wj(A)  =  Min  qiPj2+a'jb'j2 - — 

Pi2+°w, 


cp5iiA+a 2 1  )+Wi+1( 


n  2  ,  2 

Pi  +°w, 


Cp^A+O2^)) 


(3.65) 


and  the  optimal  value  of  the  cost  is 


Min  J(P0, 
p.j . pJ 


.,Pn2)  =  W  „(<) 


(3.66) 


We  next  show  that  a  solution  to  the  above  problem  always  exists.  If  we  define 


a'  2  2 

f(A,P,!)  a  q,P,2  +  Cp.ijA  +  «*  )  +  witl 

P;+<T2 


a  2((p2,A+o  2(  t ) 


P  2+a  2 

*  1  W, 


(3.67) 


Wj(A)  =  Min  f(A,P2) . 


Note  that  W;  is  a  continuous  function  of  its  argument  if  W.+1  is,  since  the  continuity 
of  Wi+1  implies  continuity  of  f.  From  the  continuity  of  WN+1,  (which  was  defined  to  be 
zero),  the  continuity  of  W;  follows  for  all  i.  We  also  note  that  as  P2— »oo,  f(A,P2)— >oo 
also,  and  since  P2^0,  the  search  for  P*2  can  be  confined  to  a  compact  set  over  which  a 
continuous  function  always  admits  a  minimum. 

The  dynamic  program  (3.65)  can  therefore  be  solved,  yielding  values  for 


P*o.P*l . P*N 


and  we  have  the  following  theorem. 


Theorem  3.4.  Consider  the  problem 


Minimize  J(hN,yN) , 

hN.yN 

subject  to  (3.1a)  through  (3.1e)  where  J(hN,yN)  is  defined  by  Equation  (3.60). 

(a)  The  optimum  policies  {h*}  and  {y*}  for  this  problem  are  given  by  (3.52)  and 
(3.58),  respectively,  using  \/s  and  A;’s  as  defined  by  Equations  (3.51)  and  (3.53),  with  the 
solution  to  the  dynamic  program  (3.65)  providing  the  optimum  power  levels,  i.e., 

p.2  =  p*2  for  i=0 . N  . 

(b)  The  optimum  cost  is  given  by 

N 

=  £  ^i2  +  24* , . 

1=0 

with  the  A^’s  being  defined  by  (3.63). 

□ 

An  Illustration 

The  optimal  power  levels  depend  critically  on  the  power  penalties  (q/s).  If  for  the 
problem  stated  at  the  beginning  of  this  section  we  assume  N=1  and  the  following  parame¬ 
ter  values 

a*  =  1.0,  <.^=1.0.^  =1.0.  ^=1.0 
q0  =  2.0  ,  q,  *  4.0  ,  p0  =  0.5  ,  a'0  =  1.0 
b'0  =  1.0  ,  a'j  =  2.0  and  b't  =  1.0  , 

then  the  optimal  value  of  the  cost  is  3.5,  which  is  attained  by  P0"=P  2=0.0.  If  the  power 
penalty  q0  is  changed  to  0.25  with  all  other  parameters  remaining  the  same,  we  can 
achieve  an  optimal  cost  of  2.9747,  which  is  attained  by  using  P02  =  1.4495  and  P,2  =  0.0. 


If  the  power  penalty  qj  is  also  changed  to  0.25,  then  the  optimal  cost  is  further  reduced  to 
1.9968,  which  is  attained  by  P02  =  1.1609  and  P2  =  1.9876.  It  is  notable  that  the  optimal 
solution  satisfies  a  threshold  property,  and  the  number  of  channels  in  use  depends  on  the 
relative  magnitudes  of  the  weighting  terms. 

3.7.  The  Infinite  Horizon  Problem 

Notation 

Let  Xj  denote  the  realization  of  a  first-order  Markov  process: 

xi+i  ~  Pxi  +  mi  •  (3.68) 

Here  x0,  m0,  mlt...  are  zero  mean  Gaussian  random  variables: 

x0  ~  N(0,ox2) 
m(  ~  N(0,a^)  . 

The  measurement  is  a  noise  corrupted  version  of  the  control  Uj: 

y^Uj  +  Wj  (3.69) 

where  w0,  w1(..  are  zero  mean  Gaussian  random  variables: 

W;  ~  N(0,a  2) 

We  are  concerned  with  obtaining  the  optimal  solution  to  Problem  P°°  below. 

Problem  P°° 

oo 

Minimize  J(h°o,y0°)  =  E[J2(qu  2+a(vi— Xj)2)/3‘] 
ha-y°°  j=o 


cu> 

Ul 
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« 
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V- 


£ 


where  a,  q  are  given  positive  constants,  3  is  the  given  discount  factor  (0<3<  1 )  and 
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(3.70a) 

(3.70b) 

□ 


Uj «  h^Xj.y1"*1) 

v;  =  yi(y‘) . 


We  treat  the  infinite  horizon  problem  as  a  limit  of  the  finite  horizon  case  with  horizon 
length  N,  as  N— »oo.  This  may  be  done  provided  that  the  discounted  cost  remains  bounded 
and  the  optimum  policy  sequence  converges  to  a  well-defined  limit. 


i 


The  truncated  version  of  the  finite  horizon  problem,  with  horizon  length  N,  is  given 
as  Problem  PN  below: 


Problem  P^ 

N 

Minimize  JN(hN,yN)  =  E[^(qui2+a(vi-xi)2)/3'] 

hN,vN  ia0 

subject  to  (3.68)  through  (3.70). 

□ 

For  notational  convenience,  let  q';  4  q3*  and  a'l  4  a/31. 

N 

We  first  consider  the  following  hard  constraint  version  of  Problem  P  . 


Problem  PHN 

N 

Minimize  E[£  a'^Vj— x,)2] 

.  N  N 

h  -y  i=o 

subject  to 
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under  the  constraint  (3.68)  through  (3.70). 


The  following  Lemma  now  follows  directly  from  the  analysis  in  the  preceding  sec¬ 
tions. 


Lemma  3.6:  (a)  Problem  PHN  admits  an  optimal  solution  which  is  linear  in  the  measure¬ 
ments  and  is  given  as  follows: 

u*j  =  h*j(Xj,y1-1)  =  X.jCXj — E(Xj  I  y1-1)) 
v*j  =  y*j(y‘)  =  X; 

where 


2  P0 
\>  =  — 


\2  = 


for  i=l . N  , 


2  .  .2 


the  A,’s  satisfy 


2  2 


n  2  .  2 

P  +o 

o  w 


n  2  2 
P  -fa 

1  l  w 


(p2Aj_1+or^)  for  i=l . N 


and  x,  =  E(x;  I  y1)  satisfies  the  recursion 


(3.71a) 


(3.71b) 


(3.72a) 


(3.72b) 


,v'  v“ 


s 

i 

§ 
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P02+ax2 

Xj  =  PX;.!  + - 1 —  y{ . 

P,!+“w 


Cb)  The  minimum  value  of  the  cost  is 


J*=£a'iV 


Now  let  Jp  denote  the  infimum  of  JN(hN,yN)  under  the  hard  power  constraints,  i.e., 

JP  £  inf  JN(hN,yN) . 

hN.yN,Eth>p,J 

Using  a  sequence  of  equalities  and  inequalities  as  in  (3.62),  we  have 


Jp=Z(qPi2+aA*i)0i 


where  the  A;’s  are  as  defined  in  (3.72),  and  A^’s  are  defined  likewise,  with  P2  replaced  by 


The  next  task  is  to  find  the  optimum  power  levels  {P;2*'sK  which  is  done  via  the  fol¬ 
lowing  deterministic  optimal  control  problem: 


The  Deterministic  N-stage  Problem 


Minimize  £  (qP;2  +  aAj)/3' 


Po . Pn  i=0 


(3.73) 


subject  to  (3.72a)  and  (3.72b). 


Notation:  For  each  positive  scalar  A,  let  {Wk(  A)}^1  be  defined  recursively  by 


vyjyvyvri 


mm 


I 


Wn+1(A)  =  0 


Wk(A)  =  inf[qP2+ - (pi+ff„)  +  /3Wk 

p3  P2+o 2 

W 


(3.74a) 


— (p2A+o  2)  ]  for  k=N,...,0  .  (3.74b) 


i  rj2  ,  2 

‘P  +aw 


Let  P  2  =  Pk  (A)  be  a  minimizing  solution  of  the  right-hand  side  of  (3.74b),  whenever  it 
exists,  and  let  {A*k}Ji0  be  the  trajectory  sequence  defined  recursively  by 


2  2 

,  °w°x  , 

A*0  =  ( - ) 

po2(ox2)+o^ 


(3.75a) 


A\  = 


Finally,  let 


P,!(AV,)+Ow 


Po'  =  Po'0 

P,:"  =  P,2(  4\_, )  ■ 


(p24*k_,  +  a  2) 


(3.75b) 


(3.76a) 

(3.76b) 


Proposition  3.1: 

(i)  The  minimization  problem  (3.74b)  admits  a  solution  for  each  positive  A. 

(ii)  The  control  problem  (3.73)  admits  a  solution  {P*2’}^  which  is  given  by  (3.76)  and 
the  corresponding  optimal  trajectory  is  generated  by  (3.75). 

(iii)  The  minimum  value  for  the  optimal  control  problem  is  W0(ax2). 

The  proof  of  the  proposition  above  follows  from  the  following  lemmata. 


Lemma  3.7:  The  value  of  the  optimal  control  problem  (3.73)  is  J*  =  W0(ox2)  where  W0(-) 
is  obtained  through  the  recursive  equations 


wn*i(a)  =0 


(3.77a  1 


Wk(A)  =  inflqP2  + - (p2A+a^)+  j3Wk+1( - (p  A  +  a  2))1  f  or  k  ^  N  .  (3.77b) 


t>2  ,  2 

P  +a„ 


n2  ,  2 
P  +aw 


Furthermore,  if  the  right-hand  side  of  (3.77b)  admits  a  solution  sequence  {Pk}, 


k  ^  N,  then 


P02(ax2) 


Pk(A*k_i)  for  1  <  k  <  N 

provide  the  optimal  solution,  where  A*k  is  generated  by  (3.75). 


Proof:  This  follows  from  a  standard  dynamic  programming  argument. 


Lemma  3.8:  For  every  A  >  0,  there  exists  a  solution  to  the  right-hand  side  of  (3.74b). 


Proof:  Define  f(A,P2)  by 


2  2 

7  7  a°W  7  7 

fk(A,P2)  =  qP2  + - +  Wk+1( - (pz A  +  a  2)) 


r>2  ,  2 

P  +Ow 


nZ  .  2 

P  +aw 


(3.78) 


so  that 


Wk(A)  =  inf  fk(A,P2)  . 


Note  that  Wk,  which  is  positive  for  all  k,  is  a  continuous  function  of  its  argument  if  Wk+1 
is,  since  the  continuity  of  Wk+1  implies  continuity  of  fk.  From  the  continuity  of  WN+1 
(which  is  defined  to  be  zero)  the  continuity  of  Wk  follows  for  all  k.  We  then  note  that  as 
P2-*oo,  so  does  f(A,P2),  and  since  P2^0,  the  search  for  an  optimum  may  be  confined  to  a 
compact  set  over  which  a  continuous  function  always  admits  a  minimum. 


JVW'  F.1ATC  JV1WI] 


The  following  lemmata  will  be  vised  in  the  construction  of  the  solution  for  the 
infinite  horizon  problem. 


Lemma  3.9:  Wk(A)  is  strictly  increasing  for  decreasing  k,  for  all  A>0,  i.e., 
W„(A)>  Wk*.(A)Vk<N. 


Proof:  Clearly  the  lemma  is  true  for  k-N,  since  WN+1  A  0  and  WN(A)  is  necessarily  larger 
than  zero  for  all  A.  We  now  note  the  following  sequence  of  equalities  and  inequalities: 


Wk(A)  -  Wk+1(A) 

2  2 

7  7  7  ®  2  7  7 

=  Min[qPZ  + - -  (p2 A  +  a‘)  +  0Wk+1( - Cp  A  +  a  2))] 

^>0  P2+a2  P2+a2 

2  2 

7  7  7  <JW  1  7 

-  Min[qP2  + - Cp  A  +  o‘)  +  /3Wk+2( - (pZA  +  a  2))] 

^>0  P2+o 2  P2+a^ 

ao  2  o  2 

^  qp2  +  (p2A  +  <^)  +  j8Wk+I( - —  (p2A  +  o  2)) 

P2+a2  P2+a2 

2  2 

2  2->  ,  ^ w 

- (p2A  +  a  2)  -  /3Wk,,( - 


-qP  + 


P2+<V 


Cp2A  +  a  2)) 


=  0(Wk+1  -  wk+2)  •  ( 


<>w(p2A+0 


P2+^w 


~2  2 
P+^w 


k 

I 

ft 

3 

$ 


J 

i 


where  P  is  chosen  as  the  argument  of  the  first  minimization.  (In  case  of  nonunique  solu¬ 
tions,  any  one  of  the  minimizing  solutions  may  be  chosen). 

Thus  if  Wk+1(A)  is  larger  than  Wk+2(A),  then  Wk(A)  is  larger  than  Wk+1(A).  Since 
Wn(A)  is  known  to  be  larger  than  WN+1(A),  the  proof  is  complete. 


Lemma  3.10:  W.  (A)  is  an  increasing  function  of  A  {W.  (A)  T  A}  for  all  k^N. 


'  •*. 
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hit1#  v1#  k*a  . 


Proof:  We  prove  this  by  induction.  First  consider  the  case  with  k-N.  We  have 


Wn(A)  =  MinfqP  + - 

**><>  V^+ol 


—  (p2A  +  ob) . 


There  are  only  two  possible  cases,  either  P2*  =  0,  or  P2*  >  0.  If  P2’  =  0,  then 
Wn(A)  =  a(p2A  +  o^).  If  P2*  >  0,  which  requires 

aa2(p2A+a2) 


(P2*+ct2)2 


1 


and  we  get 


Wn(A)  =  2a  Aq'iaw(p2A  +  a  2/*  -  qo*  . 


Thus  the  lemma  is  true  for  k-N. 


Now,  if  W.  .(A)  T  A,  then  for  each  P2 


Also, 


n2  ,  2 
P  +aw 


W 

- (p  A  +  om)  T  A 

r»2  .  2 

P  +aw 


ao2(p2A+o^) 

(qP  + - )  T  A  , 

P  +aw 


C3.79) 


(3.80) 


and  thus  both  terms  in  the  expression  to  be  minimized  to  obtain  WK(  A)  are  increasing  in  A 


V.W  >.1  "O  V*.  H.1  ■ 


B 
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for  all  P2.  Therefore,  Wk(A)  T  A. 


Lemma  3.11:  For  each  A  >  0,  Wk(A)  is  bounded  above  for  all  k,  by  an  affine  function  of 


A,  i.e.. 


0  <  Wk(A)  <  filkA  +  fi2k. 


Proof:  The  proof  is  by  induction,  using  the  observation  that  since  Wk(A)  is  given  by  the 
minimum  over  P2,  an  upper  bound  is  given  by  the  value  that  the  expression  to  be  minim¬ 


ized  attains  when  P  is  fixed  arbitrarily  at  zero. 


Thus  Wn(A)  ^  a(p2A+a^)  and  we  may  choose  Cl  j  N  =  ap2,fi2N  =  acr 


Now  consider  the  following  sequence  of  equalities  and  inequalities 

2  2 
7  aow  aw 

Wk(A)  =  Min[qP2  + - (p2 A  +  a  2)  +  0Wk+1( - (p2A  +  a^))] 

p2  P2+a*  P2+a2 

,  affW  7  7 

<  Min[qP2  + - (p2A  +  a  2)  +  0Wk+1(p2A  +  a  2)] 

p2  P2+a2 

<  a(p2A  +  ab  +  0Wk+1(p2A  +  o  2) 


^  a(p2A  +  o  2)  +  0(fllik+,Cp^A  +  o  2)  +  Q2>k+1) 


2»  ,  2, 


(ap  +  &Cllk+lp  +  aom  +  &Cllk+lo m  +  &Cl2k 


-  ^l.k^  +  ^2,k  • 


Therefore,  the  lemma  is  proved  with  the  sequences  { Q,  k}  and  { Q2k}  defined  recursively 


by 
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^l,N“  aP  >  ^2.N  — a<7m 

Qlx  =  ap2  +  0p2Cllx+1 

02>k  =  aam  +  3<7mfij  k+1  +  3^2, k+i 


We  now  return  to  the  study  of  the  infinite  horizon  problem.  Note  that  since  the 
optimal  policy  for  the  stochastic  control  problem  is  linear,  the  stationary  limiting  policy  is 


given  by 


h*n(xn-y  '  =  X*<xn  “  £(xn  1  y”  ^ 


where 


2  .  ,  ,  2 
P  A*+°m 


(3.81) 


(3.82) 


with  P  and  A*  being  obtained  through  the  stationary  solution  of  the  optimum  control 
problem  as  N-»oo. 

For  each  N,  denote  the  solution  given  by  Proposition  3.1.  (ii)  by  {Pk*}N,  k<N.  We 


then  expect  that 


P2*=  Lim  {P2’}N 


for  every  finite  k. 

To  establish  the  existence  of  this  limit,  we  recall  that  WJA)  is  strictly  increasing  for 
decreasing  k<N  (Lemma  3.9)  and  further  that  it  is  bounded  above  by  an  affine  function 
(Lemma  3.11).  This  last  property  follows  since  both  Q]k  and  are  bounded  in  retro¬ 
grade  time, 
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a  P 


a  a. 


£2lk  <  ( - )  and  ft2k  < 

1  -fr>2 


(3.83) 


(1— 0X1— p2/3) 


Hence,  Limit  Wk(A)  =  W(A)  where  the  limiting  function  satisfies 


fc- *©o 


V(A)  =  Min 
p2 


2  w 

IqP2  + - 


aojp  A+ff2) 


0W 


n2  2 
P  +aw 


p2+*i 


(p2A+o2) 


(3.84) 


Denote  the  minimizing  solution  here  by  P2(A).  Now  note  that  since  p2<l,  Equation 
(3.75b)  describes  a  stable  system  with  Pk2  replaced  by  P2,  and  hence  Ak— >A*  where  A* 
solves 


A*  — 


W  (p2A*+o  2) 


Let 


P2(A*)+a  2 


P2*==P2(A*). 


(3.85) 


(3.86) 


Then  we  have  the  following  solution  to  the  infinite  horizon  problem: 

Theorem  3.5:  With  N— »oo  in  (3.73),  the  stochastic  control  problem  in  consideration  admits 
the  optimal  stationary  policies, 

n— l  > 


h*n  =  \*(xn  -  E(xn  I  y11  *)) 


for  n  sufficiently  large,  where 


X*2  = 


with  A*  and  P2*  given  by  (3.85)  and  (3.86). 


To  numerically  compute  the  optimal  stationary  policies,  we  start  with  A0  =  0  and 
run  the  following  algorithm: 


Algorithm  A: 


(1)  Compute 


aa,jJ(p2Ak+a  2)  /3(qP2+aAk) 


-  ,  “n ' "m' 

(Pk r  =  arg  minfqP  + - + 

P2+a^ 


(1-3) 


(2)  Compute  the  new  value,  Ak+1,  by 


Vi® 


(P*)*+°w 


C  p\  +  0 


(3)  Go  to  step  (1),  and  iterate. 


The  optimal  cost  is  then  given  by 


W(A*)  =  (q(P2)*  + 


ao  2(p2A*+a  2) 


■)/(l  -3). 


(P2)*+o^ 


We  next  show  that  Algorithm  A  always  converges.  We  first  note  that  (Pk  )*,  found 
from  step  (1)  of  the  algorithm,  satisfies 


*6/ . _ py/z 

(Pk  )*  =  Max{0,(- - - -  ow(p2Ak+a  2)Vj  -  a  2)} 

q'A 


which  implies  that  if  Ak+1  >  Ak,  then  (P2+J)*  ^  (Pk)*. 


Now,  given  Ak  >  Ak_,,  we  have 
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Ak+i  \ 

2  2 

°v  (  2*  .  2\  ,  2.  L  2, 

- - (p  Ak+am) - (p  Ak_i+am) 

CPkl)*+<d  CPk2_P*+a 2 

aw  «  ■%  aw 

>  - (p2Ak+a^) - (pVj+uJ 

(Pk2)*+a2  CPk2)*+a^ 

aw  , 

- - p2(Ak-Ak_1) 

(Pk)*+a  2 

>  0. 


Therefore,  since  Aj  >  A0,  it  follows  that  the  Ak’s  form  a  monotone  increasing 
sequence.  Further,  to  show  that  the  Ak’s  are  bounded  above,  we  consider  the  sequence 

r0  =  o 

T-  2—  ,  2 

^k+1  -  ^  ^  +  °m’ 

and  note  that  if  Tk  >  Ak,  we  have 

rk+1  =  P2rk  +  a2 

>  P\  + 

°  W  2  2 

>  - <P2Ak  +  0 

(Pk  )*+o  2 


i.e.,  if  rk  ^  Ak,  then  Tk+1  ^  Ak+1.  But  T0  =  A0(=  0),  and  the  sequence  rk  is  bounded 
above  by 
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Therefore,  the  monotone  sequence  Ak  is  also  bounded  above,  and  the  convergence  of  the 
algorithm  follows. 


Tables  3.1  through  3.4  provide  a  few  of  these  convergence  results  for  given  parame 
ter  values. 


TABLE  3.1.  CONVERGENCE  RESULTS  FOR  THE 
a«o*=o2=  1.0,  p2  =  0  =  q  =  0.5. 


Iter.  # 
0 
1 
2 

3 

4 

5 

6 

7 

8 
9 


PROBLEM  WITH 


A 

(/,2)‘ 

0.0 

0.0 

1.0 

0.22474 

1.22474 

0.26979 

1.26979 

0.27863 

1.27863 

0.28035 

1.28035 

0.26069 

1.28069 

0.28076 

1.28076 

0.28077 

1.28077 

0.28077 

1.28077 

0.28077 

TABLE  3.2.  CONVERGENCE  RESULTS  FOR  THE  PROBLEM  WITH  a  =  a„  =  1.0. 
p2  =  3  =  0.5,  q  =  =  0.1. 


Iter.  # 
0 
1 
2 

3 

4 


L 

C P 2)‘ 

0.0 

0.60710 

0.14142 

0.63167 

0.14633 

0.63251 

0.14650 

0.63254 

0.14650 

0.63254 

Wv 


?2 


.  A-:_  •  A-  t*.  *'»•*  t  x  <>.i  -.•«  ,j->  *  .  ***  4 


TABLE  3.3.  CONVERGENCE  RESULTS  FOR  THE  PROBLEM  WITH  a»<y„-1.0. 
p2  =  &  =  0.5,  q  =0.1.  o*  =2.0. 


Iter.  It 
0 

_1 _ 

2 

3 

4 

5 

6 

7 

8 


4 

(PV 

0.0 

1.16227 

0.63245 

1.62798 

0.72559 

1.69161 

0.73832 

1.70021 

0.74004 

1.70138 

0.74027 

1.70153 

0.74030 

1.70155 

0.74031 

1.70156 

0.74031 

1.70156 

TABLE  3.4.  CONVERGENCE  RESULTS  FOR  THE 
p2  =  a2  =  6  =  0-5,  a  =  =  1.0,  q  =  0.1. 


Iter.  # 
0 
1 
2 

3 

4 

J _ 

6 

7 


PROBLEM  WITH 


4 

(PY 

0.0 

0.58113 

0.31622 

0.81399 

0.36279 

0.84580 

0.36916 

0.85010 

0.37002 

0.85069 

0.37013 

0.85076 

0.37015 

0.85078 

0.37015 

0.85078 
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We  finally  note  that  an  infinite  horizon  version  of  the  originally  formulated  Problem 
PI,  can  be  solved  via  an  equivalent  problem  of  the  form  P°°. 

Consider  the  problem  of  minimizing  J(h“,-y°°)  where 


J(h“,y“)  =  E[£  3‘(qu2  +  a (5x*+l  +  bv2)] 


subject  to  (3.1b),  (3.  Id)  and  (3.1e),  where 


xi+i  =pxi  +  mi-vi. 


Using  (3.4)  and  (3.5),  we  find  that  the  cost  can  be  rewritten  as 


J  =  z  qi3‘u  2  +  £  a'^Vj— b'jXj)2  , 


where 


a'i  -  /3‘(b+kj3)  =  a'& 


t 

i* 

* 


with  the  k  found  by  solving  for  the  positive  root  of  the  equation: 


i.e.,  the  k  satisfies 


(k— a)(k3+b)  =  kb/3p2 , 


k  =  -i_ (V(b—  b0pz— a3)2+4ab3  —  (b— b3p2-a3))  • 
23 


Thus,  an  infinite  horizon  version  of  the  originally  formulated  problem,  with  discounted 

oo  7 

cost,  may  be  solved  by  solving  a  problem  of  the  form  P  ,  with  a  replaced  by  alb')". 
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3A  Conclusion 


ft 

ft 


In  this  chapter  we  have  studied  the  problem  of  simultaneous  communication  and 
control  for  first-order  ARM  A  models  with  feedback.  These  are  stochastic  team  problems, 
where  the  design  of  the  measurement  strategy  itself  is  a  part  of  the  problem.  As  such, 
they  are  difficult  to  analyze  because  of  the  nonclassical  nature  of  the  information  struc¬ 
ture.  For  cases  with  hard  power  constraints  on  the  measurement  strategies,  we  have 
shown  that  the  optimum  measurement  policy  is  to  linearly  amplify  the  innovation  at  each 
stage,  to  the  maximum  permissible  power  level.  For  the  cases  with  soft  power  constraints 
the  structure  of  the  solution  is  similar;  however,  now  the  optimal  design  of  the  power 
levels  is  also  a  part  of  the  problem.  These  optimum  power  levels  may  be  found  via 
dynamic  programming.  We  have  then  studied  some  infinite  horizon  stochastic  team  prob¬ 
lems  involving  a  first-order  ARMA  model,  established  the  existence  of  optimal  stationary 
policies  for  these  problems,  provided  an  algorithm  that  always  converges  to  the  optimal 
solution,  and  also  provided  some  numerical  examples  illustrating  the  calculation  of  these 
stationary  policies. 
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CHAPTER  4 

SIMULTANEOUS  COMMUNICATION  AND  CONTROL: 

GENERAL  ARMA  MODELS  WITH  FEEDBACK 

4.1.  Introduction 

In  this  chapter  we  study  the  problem  of  simultaneously  designing  communication 
and  control  strategies  for  problems  involving  ARMA  models  of  orders  higher  than  one.  In 
Section  4.2  wt  formulate  the  general  problem  and  then  the  transformed  problem  consist¬ 
ing  of  squared  differences.  In  Section  4-3  we  study  one  of  the  simplest  such  problems 
involving  a  second-order  ARMA  model,  and  show  that  the  strategies  which  are  optimal 
over  the  linear  class  may  be  outperformed  by  appropriately  chosen  nonlinear  strategies. 
In  Section  4.4  we  consider  optimality  over  the  affine  class,  and  show  that  within  this  class 
the  optimal  policy  consists  of  transmitting  only  the  current  innovation,  multiplied  by  a 
gain  factor.  In  Section  4.5  we  study  the  linear  solutions  for  the  second-order  ARMA 
model;  this  illustrates  the  methodology  and  concepts  for  the  more  complex  problem 
involving  the  general  model  which  is  treated  in  Section  4.6.  The  concluding  remarks  in 
Section  4.7  then  end  this  chapter. 

4.2.  Problem  Formulation 

In  Chapter  3  we  had  studied  a  stochastic  dynamic  system  involving  a  first-order 
ARMA  model,  with  the  current  state  directly  correlated  only  with  the  immediately 
preceding  state.  In  case  we  allow  this  correlation  to  extend  to  j  previous  stages,  we  obtain 
a  jth-order  ARMA  model.  Accordingly,  let  us  suppose  that  the  stochastic  system  is 
specified  by  the  following  set  of  equations: 
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j-1 


Xi+1=  I  Pi+l.i-ltXi-k  +  mi  —  Vt 
k=0 


(4.1) 


and 


y,  =  ui  +  wi 


(4.2) 


along  with 


Uj  =  hjCXj.y1-1) 


(4.3) 


vi  =  Yi(y‘)  • 


(4.4) 


Here  (4.1)  and  (4.2)  are  the  state  and  measurement  equations  respectively,  the  ran¬ 
dom  variables  x0,  w;  and  m(  (i^O)  are  assumed  to  be  independent,  zero-mean  and  Gaus¬ 
sian  with  variances  o(2,  (the  subscript  being  the  identifier),  and  pk  J  =  0  for  j<0.  The 
functions  h;  and  -y i^O,  are  the  communication  and  control  policies,  respectively,  each 
Borel  measurable  in  its  arguments,  and  leading  to  second-order  random  variables  u;  and 
v.,  respectively. 


As  seen  in  Chapter  3,  we  may  formulate  a  hard  constraint  version  of  the  problem,  by 
restricting  the  communication  policies  to  satisfy 


E[u,2]  <  P‘ 


(4.5) 


or  the  soft  constraint  version  may  be  formulated,  using  the  additional  term 


Etlq.u,2] 


i.6) 


in  the  cost  functional,  implying  that  a  tradeoff  between  higher  signalling  costs  and  lower 
estimation  costs  is  permissible. 
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These  soft  and  hard  constraint  versions  are  given  below  as  Problems  PS°  and  PH°, 


respectively. 


Problem  PS0 


Minimize  E[£  q^2  +  ci+1xi^.l  +  d;V  2] 


.  N  N 

h  .y  i=0 


subject  to  (4. 1 )  through  (4.4). 


Problem  PH° 


Minimize  E[£  c^jX^  +  d;V2] 


.  N  N 

h  -y  i=o 


subject  to  (4.1)  through  (4.5). 

Using  completion  of  squares  and  a  redefinition  of  the  v^s,  (as  in  the  case  of  the  first- 
order  ARMA  model)  we  may  obtain  th”  equivalent  problems  PS  and  PH  below: 


Problem  PS 


j 

Minimize E[£  qjU2  +  a^Vj  —  £  bj i_kxi_k)2] 


.  N  N 

h  y  i=o 


subject  to  (4.2)  through  (4.4),  with  xi  replaced  by  x-  and  vi  replaced  by  v;, 


Problem  PH 


Minimize  E[ai(vj  -  £  bii_kxi_k)  ] 


subject  to  (4.2)  through  (4.5),  with  x(  replaced  by  x;  and  vt  replaced  by  v.  For  these 
transformed  problems,  the  x/s  are  given  by  the  following  recursion: 


l 


MViWJVfi’nv 


'8 


,  urwvw^rKViri . _ _ _ 


*i+l  —  X  Pi+l.i-k^i-k  +  mi 

k-0 


(4.7) 


where  x0=x0  is  given.  pk }  =  0  for  all  j<0  and  bk  j  =  0  for  j<0.  Furthermore,  the  precise 
(recursive)  expressions  for  the  a/s  are  given  in  Appendix  A,  where  the  details  of  the 
justification  for  this  reformulation  can  also  be  found. 

We  now  turn  to  analyzing  these  reformulated  stochastic  team  problems. 


4.3.  Nonoptimality  of  Linear  Laws 

In  this  section  we  show  that  for  one  of  the  simplest  team  problems  of  the  type  above, 
involving  an  ARM  A  model  of  order  2,  the  optimum  linear  solution  may  be  outperformed 
by  an  appropriately  chosen  nonlinear  policy. 

We  first  restrict  our  attention  to  the  following  stochastic  team  Problem  P4,  of  which 
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a  schematic  representation  is  provided  in  Figure  4.1. 


8 

C> 


E(x  i  v  1.y2)=v 
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Problem  P, 


Minimize  E[(x— v)2] 

yi-yj.S 


where 


v  =  S(yi,y2) 


(4.13) 


subject  to  the  hard  power  constraints: 


E(Uj2]  <  P  2 
E[u  2]  <  P22 


(4.14a) 

(4.14b) 


Note  that  if  the  problem  involved  estimating  Xj  at  the  decoder  (instead  of  x),  then 
we  would  have  had  the  two-stage  version  of  a  problem  involving  a  first-order  ARMA 
model  as  studied  in  Chapter  3,  for  which  the  optimal  solutions  have  been  shown  to  be 
linear. 

We  now  show  that  Problem  Pa  above  does  not,  in  general,  admit  an  optimal  linear 
solution.  This  is  done  by  constructing  an  instance  of  the  problem  where  the  optimal  linear 
strategies  are  outperformed  by  appropriately  chosen  nonlinear  strategies. 

In  order  to  see  why  one  might  suspect  nonoptimality  of  affine  laws,  consider  the 
above  problem  with  a*  =  G.  We  then  have  Problem  P'a  below  which  is  represented 


1 

Xj  =  x  +  nj 

(4.8) 

I 

c 
»— » 

II 

*-* 

N 

X 

t 

(4.9)  : 

Yi  ~  uj  +  Wj 

(4.10)  I 

t 

u2  =  y2(xi>yi) 

(4.11)  ; 

a 

y2  *  u2  +  w2 

(4.i2)  ; 

^  and 

s 

schematically  in  Figure  4.2,  and  for  which  the  hard  power  constraint  on  u2  is  immaterial 
since  there  is  no  noise  to  combat. 


Problem  P'4 


Minimize  E[Cx— v)2] 

subject  to  (4.8)  through  (4.11),  and  (4.13),  along  with  the  restriction 

y2  =  u2-  (4.15) 

□ 


Note  that  since 


v  =  S(yj,y2) 


(4.16) 


where 


y2  =  u2  =  y2(xi,yi) 


n 


l 


x 


E(x  ly  j.y2)=v 


Figure  4.2.  Schematics  for  Problem  P'  . 


we  may  equivalently  write 


v*8(yl,y2(xi,yi))  (4.17a) 

=  S'(y1,x1),  (4.17b) 

and  thus  Problem  P'a  may  equivalently  be  represented  as  in  Figure  4.3  below. 

We  thus  obtain  a  problem  of  simultaneously  designing  encoding  and  decoding  policies 
with  side  information  at  the  decoder,  for  which  nonlinear  strategies  that  outperform  the 
optimal  linear  strategies  do  exist  (see  Appendix  B). 

Since  linear  policies  are  not  optimal  for  Problem  Pa  with  =  0,  they  may  continue 
to  be  nonoptimal  for  small  enough  values  of  We  show  next  that  this  is  precisely  the 
case.  In  particular,  if  we  consider  the  optimal  linear  design  for  Problem  Pa,  using 

ui  =  Yi(x)  -  *-ix 

u2  =  Y2(xi-yi)  =  Mxi  -  E(xi 1  yi)) 

where  \j  and  \2  are  chosen  to  meet  the  hard  power  constraints  with  equality,  (this  being 
the  optimal  choice  in  the  affine  class,  as  to  be  shown  later),  we  have 


x2  -  E(xt  I  y2)  =  iij  + 


x  — 


W, 


n  2  2 

Pl+<Tw, 


n  2  ,  2 

Pt+aw, 


which  implies  that 


ii2  =  X2 


\  2 


X  — 


w  +  n 


r,  2  .  2 

P1  +°w, 


n  2  2 

Pl+0w, 


with 


P22(Pl+<) 


ffx2aw1+anI(Pie+°w1) 


The  mean  square  error  in  estimating  x  from  the  simultaneous  observation  of  y 
then  is 


o  2a  2 

E[(x  -  E(x  I  y1,y2))2]  = - - - 

(P’+a’XP  l+ol) 

Considering  the  situation  with  ox2  =  100.0,  on2=0.99,  =1.0,  a ^ 

P  2  =  85.0423  and  P22  =  100.99,  we  find  that  the  optimal  linear  policy  yields  t 
0.53467. 


<  + 


n  L  2 

P,  On 

2  n  • 


(a  2+o  2o  2  /(P,2+a  2  )) 


(4.19) 


i  and  y. 


(4.20) 


=  0.01. 

cost  of 


We  next  consider  the  design 
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•y^x)  =  x  +  esgn  x 

y^x^sxj 

and 

(y1+y2-«)/2  if  y2  >  0 

s(yi.y2)  = 

yj+y2+€)/2  if  y2  <  0 

(letting  €  =  — 1.0  we  obtain  Efu^]  =  85.0423  and  E(u2  )  =  100.99). 

Now 

y2  =  x  +  nt  +  w2 

=  X  +  W3 

where 

w3  ~  N(0,1) . 

If  we  calculate  the  mean  square  error  under  the  above  policy  by  an  analysis  similar  +o 
that  used  in  Appendix  B,  we  find  that  the  nonlinear  policy  yields  a  cost  of  0.53172,  and 
hence  is  superior  to  the  optimal  linear  policy. 

We  now  return  to  the  problem  of  showing  nonoptimality  of  linear  laws  for  at  least 
some  instances  of  higher-order  ARMA  models.  Consider  the  following  second-order  model 
with  feedback,  illustrated  in  Figure  4.4.  We  have 

X2  =  P2!X1  +P2oxo+m! 

X1  =  Pl0X0  mo 


and  x0,  m0  and  m,  are  given  independent,  zero-mean,  Gaussian  random  variables. 


I 


Figure  4.4.  Schematics  for  a  second-order  model  with  feedback. 

The  problem  is  to  minimize  E[(v2— x2)2]  under  the  schematics  of  Figure  4.4,  with 


v2  =  Y2(y0'yvy2)  • 


Let  us  now  suppose  that  is  arbitrarily  large,  essentially  making  the  third  channel 


redundant,  and  therefore 


E(x2|y0.yl,y2)  =  H(x2|yo.yI) 


Further  suppose  that 


p2i  —  0  ,  p20  —  1  .  °m  —  0  ,  p,0  —  I 


which  imply 


x2  X0 

X1  *  x0+  ml 


and  we  obtain  the  problem  depicted  in  Figure  4.5.  We  thus  obtain  a  problem  of  the  type 


rT’iVri'O'.V.V.y .  V.V 


S* 


Figure  4.5.  The  second-order  ARMA  model  under  the  given  restrictions. 

P4,  discussed  earlier  in  this  section,  for  which  there  are  instances  when  linear  strategies 
are  not  optimal.  Hence  we  see  that  there  are  instances  of  the  general  problem  (described 
by  second-order  ARMA  processes)  for  which  optimal  linear  designs  are  not  optimal  in  the 
general  class  of  policies. 

4.4.  Optimality  over  the  Affine  Class 

In  the  preceding  section  we  have  shown  that  for  ARMA  models  of  order  higher  than 
one.  the  strategies  which  simultaneously  provide  optimum  communication  and  control  are 
not  necessarily  linear.  In  this  section  we  show  that  if  we  confine  the  design  to  the  affine 
class,  then  the  optimal  communication  strategies  use  a  linear  transformation  on  the  inno¬ 
vation  process. 

Theorem  4.1.  Consider  the  general  formulation  of  the  Problem  PS  given  in  Section  4.2,  but 
with  hj  resticted  to  the  class 

uj  =  hj(  Xj.y 1 )  =  LjCxj.y1- 1 ) 


where  L(  is  a  general  affine  mapping.  Then  one  may,  without  loss  of  generality,  be 


confined  to  optimizing  over  the  class  of  transmitter  policies  which  satisfy  the  structural 
restriction 

Uj  XjCXj  —  E(Xj  I  yi_1 ))  .  (4.21) 

Thus  it  is  sufficient  to  optimize  over  the  class  of  policies  which  use  a  linear  transformation 
on  the  innovation  in  xr 


Proof.  Note  that  over  the  affine  class  we  may  write 


ui  =  LjCxj.y1-1)  =  U;  +  p; 


with 


Uj  =  A-jCxj  —  E(xj  I  y1  *)) 


and 


p.-L^Cy1-1) 

where  Lt  is  an  arbitrary  affine  mapping,  and  u;  and  p(  are  uncorrelated.  Thus,  we  have 


Min  J(hN,yN)  =  E 

Vn 


=  E 


>  E 


N  j-1  j-1 

+  ajCEC  £  bu_kXi_k  I  y‘)  -  £  bitj_kXj_k)2) 

j=0  k=0  k=0 

N  j-1  j-1 

£(qiUi2  +  qiPi2  +  ajCEC  £  bs  I  y')  -  £  bi>i-kxi_kJ2) 

i=0  k=0  k=0 

N  j-1  j-1 

£  qju2  +  aj(E(  £  b-  i_kxi_k  I  y1)  -  £  b^x^)2) 


i=0 


k=0 


k=0 


We  now  note  that  the  sigma  field  generated  by  y'  is  the  same  as  the  sigma  field  generated 
by  y'  where 
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ys  =  uf  +  Wj 

=  Vi  -  Pj 


since  p;  is  y1  1  measurable. 


Hence, 


j-i  .  j-i 


Min  J(h  ,y  )  >  E  £(q,u,  +  a,(EC  £  ^  I  y')  -  I  b^x^)2) 

Yn  i*0  ksO  k=:0 


and  therefore  the  cost  functional  may  be  optimized  under  the  structural  constraint  (4.21). 


4.5.  Optimal  Linear  Strategies  for  Second-Order  ARMA  Models 

We  now  concern  ourselves  with  the  design  of  communication  and  control  strategies 
which  simultaneously  optimize  over  the  linear  class.  In  order  to  facilitate  an  understand¬ 
ing  of  the  concepts  and  methodology,  we  restrict  ourselves  in  this  section  to  a  study  of 
second-order  ARMA  models.  The  general  ARMA  model  will  be  studied  in  the  next  sec- 


(i)  The  problem  with  Hard  Power  Constraints. 


Problem  PH 


Minimize  E[£  ai(vi-bi>ixi-bi>i_1xi_1)2] 


.  N  N 

h  y  i=o 


where 


ui  =  hjCXj.y 1  ') 

v^y.Cy1) 

Elu2]  =  p  2 


and  the  xt's  are  generated  via 


AiWKmYi.VVv: 


WWW 


X1  =  Pioxo  +  m0 

$2  =  P2i^i  P20x0  mi 

Xi+1  =  Pi+l.lXi  +  Pi+l,i-lxi-l  "*■  mi 

XN+1  =  PN+1,NXN  +  Pn+1,N-1XN-1  +  mN 


Notation 


Aij  =  E[(Xj— E(xj  I  yj))2] 

A,4  S  E[(Xj— E(x,  I  yj))(xk-E(xk  I  yj))] . 


(4.22a) 

(4.22b) 


Theorem  4.2.  The  encoding  and  decoding  policies  for  problem  PH  ,  which  are  optimal  over 


the  linear  class,  are  given  by 


ui  =  yA-y1  =  XjCXj— ECxj  I  y'  ‘)) 

Vj  =  s^y1)  =  bj  jE(Xj  i  y')  +  b^Etf^  I  y‘) 


and  the  optimal  cost  is 


where 


Zai(bi>ii  +  bi^,i,L,+2bubu.1i,‘_1) 

i=0 

E[xj  I  y ']  =  E[xt  I  y'-1]  +  E[x,  I  y,] 

E(xj  I  y'_1]  =  pj  j_,H[xj_1  I  y1'1]  +  p(  ,_2E[xi_2 1  y,_1] 
E(Xj_,  I  y‘]  =  Elx,_,  I  y  "'l  +  E[x,_,  I  y,] 


E(x,  I  y,)  = 


A'  ‘ 


n  2  2 

P,  +aw. 


(4.23) 

(4.24) 


(4.25) 


(4.26) 

(4.27) 


(4.28) 


(4.29) 
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.  \(Pi,i-lAi-l  +Pi,i-2^i-l.i-2^ 

Etxj_,  I  ys]  = - Yj 


n  2.  2 

Pi  +  °w, 


where 


P 

^2  =  — 


and  the  A’s  are  generated  recursively  by 


A;‘  =  A.‘ 


°w 

1-1  1 


n2.  2 

P;  +o„. 


.  i— 1  2  .  i— 1  .  2  .  i— 1  ,  -  .  i— 1  ,  2 

Ai  “Pi,i-lAi-l  +  Pi.i-2Ai-2  +  2pi,i-iPi,i-2Ai_1  .i—2  +  am,_, 


Ai_t  =  A;_J  (Py— i^i— 1  +  Pi,i-2Ai-l  ,1-2) 

CP,2+°i> 


Ai.i-1  ~  ^Py-l^i-l  +  Pi.i-2^i-l,i-2^ 


n  2.  2 

P,  +aw 


with  the  initial  condition 


Ao  '  =  °x2n  :  Ao°-i  =  0  and  A-°i  =  0 


(4.30) 


(4.31) 


(4.32) 


(4.33) 


(4.34) 


(4.35) 


Proof.  Expressions  (4.23)  and  (4.24)  are  immediate  (since  we  know  that  the  optimum  u-  is 


linear  in  the  innovation  and  the  conditional  expectation  minimizes  the  mean  square  error), 


and  (4.25)  follows  from  the  definition  of  A.  Expressions  (4.2b)  and  (4.28)  are  due  to  the 


fact  that  y(  is  independent  of  y'  and  (4.27)  follows  from  the  definition  of  xr 


rr 
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Now, 


yt  =  A-jCxj  —  E(x j  I  y1"1))  + 


ECxj  I  yj)  =  E(Xj  -  E(xj  I  y1  j  I  ys) ; 


hence  we  have  (4.29),  and  (4.31)  follows  from  the  hard  power  constraint  on  E[u2]. 


To  obtain  (4.30)  note  that  for  any  two  zero-mean  Gaussian  random  variables  Zj  and 


z2,  we  may  write 


E[z1z2] 


E[z2] 


z2  +  n 


where  n  is  zero-mean,  Gaussian,  and  independent  of  z2. 


Thus  we  have 


y.  =  -  e( Vi 1  y'  » 


+  \Pu_2(xw  -  E(xj_2  I  y*  ))  +  \jinj_j  +  w, 


and  writing 


A  i_1 

(Xj_2  E[Xj_2 1  y1-1])  =  ■  -  —  (Xj_,  -  E(Xj_j  I  y1-1))  +  n  , 

A  i_1 


(with  n  independent  of  (Xj_j — E(xi_1  I  y1  l)))  we  have 


j_2 

Yi  =  Xj (pu_,  +  pu— 2-— —  Xxi— !  -  E(xj_,  I  y‘-1))  +  n' 
.  i-l 


and  therefore 


I 
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E[Xj_i !  ys3  =  ElXj,!  -  EtXj.!  I  y1-1)  I  yf] 


J 


is  given  by  (4.30). 

What  remains  to  establish  now,  is  the  validity  of  the  recursions  for  the  A’s. 
We  have 

Aj1  =  E[(Xi  -  E(Xj  I  y1))2] 

=  E[(Xj  -  E(xj  I  y1_1)  -  E(X;  I  y;))2] 

=  A;1-1  -  ElECxj  I  y;)2] 

and  using  (4.29),  (4.32)  follows.  Further, 

A,1-1  =  E[(x;  —  E( x j  I  y1-1))2] 


and  using  the  definition  of  xf  we  have  (4.33). 


Ajl_j  =  El(Xj_i  -  E(Xj_j  I  y1))2] 

=  E[(x;_j  -  E(Xj_j  I  y1-1)  -  E(x;_j  I  y;))2] 
=  Ajl'1  -  E[E(Xj_j  I  yf)2] 


_  A  1-1 
=  Aj — !  - 


i.e.,  we  have  (4.34).  Finally, 


A‘H1  =  E[(Xj  -  E(x;  I  yOXXj.!  -  E(Xj_j  I  y'))J 

=  E((Xj  -  E(Xj  I  y1-1)  -  E(xjl  y^XXj.j  -  E(Xj_j  I  y'_1 )  —  E(X;_i 1 
=  Ai!ni  “  ElHCxj  I  yi)E(xi_1  i  y^j 


A^Jj  =  E[(Xj  -  E(Xj  I  y1  'iXXj.,  -  E(Xj_!  I  y1-1 ))] 


.  i-l  ,  .  i-1 

—  Pi,i-lAi-l  +  9i.i-2Ai-l,i-2  ' 


along  with 


•Mi 


& 

$ 

» »< 

>{» 

>y» 


1; 

$ 

$ 

$S 


I 


hv 

a 


?4 

a 


8 

■I 


X.2Ai-1(p- .  ,A'  Z+p  .A1.1.  ,) 

'vi  “i  v^i»j — 1  “j — 1  **1,1— 2  i— l,i— 2' 


E[E(Xj  I  y;)  E(xi_1 1  y;)]  = 


n  2  ,  2 
P,-  +  a„ 


r,  2  .  2 

P;  +  <*», 


^Pi,i— l^i— 1  +  Pi,i-2^i-l,i-2^ 


implies  (4.35). 

The  initial  conditions  for  the  above  hard  constraint  version  are  immediate,  and  it  is  a 
straightforward  task  to  verify  that  using  the  given  recursions,  all  expressions  required  for 
the  solution  to  PH2  can  be  generated. 


(ii)  The  Soft  Constraint  Version 

It  follows  from  the  solution  to  the  hard  constraint  version  (as  in  the  case  of  the 
first-order  ARMA  model)  that  Problem  PS2  may  be  solved  via  the  following  nonlinear 
optimal  control  problem. 


Problem  NL 


Minimize  £  (q^  +  afr-A-'  +  aibj-i_1AiL1  +  2aibiflbu_1Ai'ii_l) 


- Pn  i=0 


where  the  A’s  are  generated  recursively  by  (4.32)  through  (4.35). 

The  solution  to  Problem  NL2  is  given  by  the  following  dynamic  program: 

Wn+1(A,;A2;A3)  =  0 
Wj(A1;A2;A3) 


Mw 

=  Min[qiP2  +  aib2( - -) 

p.a  P;2+a 2 


2  ,  Pi  ^Pi.i-lA2+Pi.i-2A3)  , 

+  aibifi_1(A2  -  — — - ) 

(P2+a2)  A, 


W 

+  2aibi.ibu-l( - — Xpu_,A2  +  Pj.i— 2A3^ 


P2+o2 

a„2  P:2 


+  Wi+1Cp2  lfi - —  A,  +  Pi+j.HjCAj  - 


(Pi2+^w)Ax 


^Pi,i— 1 A2  +  Pi.i— 2A3)  ) 


+  2Pi+l,iPi+l,i-l^Pi.i-lA2  +  Pi,i— 2A3)' 


2  ,  2 
Pi  "^0w, 


Ai^w,  (Pu-1A2+Pi,i-2A3)°, 


P2+ai 


■). 


„2,  2 

Pi  +  aw, 


Further,  the  optimal  value  of  the  cost  under  linear  policies  is  W0(ox2;0;0). 


4.6.  Optimal  Linear  Strategies  for  the  General  ARMA  Models 
(i)  The  Transformed  Problem  with  Hard  Power  Constraints 
Problem  PHJ 

N  j-1 

Minimize  E[£  a^V;  -  £  bj  j_kxi_k)‘'] 

.  N  N 

n  .y  i=0  k=0 


where 


Uj  =  hjCxj.y1-1) 

V;  =  y^y') 

E[Uj2]  =  P2 

y.  =  Uj  +  Wj 

and 

j 

*i  =  I  Py-k*i-k  +  mi-i  • 

k=l 

Theorem  43.  The  encoding  and  decoding  policies  for  Problem  PHJ,  which  are  optimal  over 
the  linear  class,  are  given  by 

ui  =  yjCXi.y1-1)  =  XjCxj  —  E(Xj  I  y1  1 ))  (4.36) 

j-1 

Vj  =  SjCy1)  =  £  bi.i-kE[Xi_k  I  y'l  (4.37) 

k=0 

and  the  optimal  cost  is 

N  j-1  j— 1 

J*=  Z  ai(  H  £  bi.i-kbi.i-m^i-k.i-m^  (4.38) 

i=0  k=0  m=0 


(4.39) 

(4.40) 

(4.41) 


(4.42) 


and  the  A’s  are  generated  recursively  by 


»  i_1  2 


_  2  2 

R  +a„ 


(4.44) 


]  j 


Ai1-1  =  +IZ  Pi. i-kPi.i-mAi-k,i— n. 

k=lm=l 

P2  j 

A-  =  A.1  1 - - - (  £  Pu-A-k.m)2  * 


m  m 


CP’+o^A/-1  k=i 


ri 

A-1-  =  Ai"1 - - - C  z  Pi.i-kA^.nX  z  Pi.i-kA^) 


n.xn  n.m 


(p,!+°*)  *-■ 


k=l 


J  ) 


A„‘  =11  Pn.n-kPn.n-pAn'-k.n-p  +  am„_  O-j^i-l  ) 

k=lp=l 


(4.45) 


(4.46) 


(4.47) 


(4.48) 


An.m  =  I  Pn.n-kA‘-k.m(i“j^m<n<i_l) 


i— 1 


k=l 


(4.49) 


with  A0  1  =  (Tx2;  Ak  ^  =  0  for  either  k<0  or  m<0. 


Proof.  The  proof  follows  in  a  manner  similar  to  that  for  the  case  j-2  discussed  in  the  pre¬ 
vious  section. 


□ 


(ii)  The  Transformed  Problem  with  Soft  Power  Constraints 

It  follows  f rom  the  solution  to  the  problem  with  hard  power  constraints  that  Prob¬ 
lem  PSJ  can  be  solved  via  the  following  nonlinear  optimal  control  problem: 
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N  j- 1  j-1 

Minimize  £  I  L  bu-kbu-nA-k.i-m) 

P<? . ?n  no 


k*0  m=0 


where  the  A’s  are  defined  via  (4.44)  through  (4.49). 


The  solution  to  the  above  optimal  control  problem  may  be  obtained  via  the  following 
dynamic  program. 


WN+J  =  0 


^i(Au  >  •  •  •  <  Ai,i-(j-i)'Ai-i,i-i . Ai-i,»-(j-i)'  •  •  •  ,  Ai-(j-i),i-(j-i)^ 

j-i  j-i 

h.  .  h.  . 

-p'-  i— k,i— p 


=  Min 

D  2 


q^+a^E  E  b,,-kb 


k=0  p=0 


J  ) 

i-1 


(  Z  Ai.i-nAi-n,,-k^  Z  Pi.i-mAi-m,i-p^ 


m=l 


+  ^i+l^U’  •  *  •  ’  Ai.i— (j— l),Ai— 1 . Ai— l.i— (j— 1)’  ’  •  ’  ’  Ai-(j-l).i-(j-l)^ 


where  the  relationship  between  the  A’s  is  as  given  in  (4.44)  through  (4.49).  The  optimal 


value  of  the  cost  is  W0(ox  ,0,0, ...,0). 


(iii)  Solutions  to  the  Original  Problem 


The  optimal  linear  solutions  to  the  original  problems  PH°  and  PS°  can  be  generated 
once  the  solutions  to  the  transformed  problems  are  available.  We  indicate  how  this  may 
be  done  for  the  hard  constraint  version.  For  the  problem  with  soft  power  constraints,  the 
procedure  is  identical,  except  that  we  first  need  to  find  the  optimal  power  levels  by  solv¬ 
ing  the  nonlinear  optimal  control  problem  via  dynamic  programming. 

Noting  that  since  Vj  is  y1  measurable,  we  have 

Xj  -  E(x(  I  y‘-‘)  =  Xj  —  E(x,  I  y‘  ‘) 


-Oh 


yy'j 


8 


$ 

K* 


jX 

ft 


a 


W 


3 


N 


-V 

:s 


and  hence  the  optimal  Uj’s  are  given  by 


u*i =  Y*(xi.y'-1)  =  XjCx-ECxj  I 


with  the  X’s  being  the  same  as  those  for  the  transformed  version.  To  obtain  the  optimum 
v/s,  note  that 

j~i 

v*i  =  S*(y')  =  Z  bi,i-kHXi  I  y1) , 

k=0 


and  recalling  that  (see  Appendix  A) 

Xi— k  =  Xi-k  —  vi — k — 1  1 


we  have 


j-i  j-i 

v*  =  I  bu_kE(x,_k  I  y‘)  -  z  Vk-i . 

k*0  k=0 

where  the  v’s  are  defined  in  terms  of  the  preceding  v*’s  as  in  Appendix  A  and  the  expected 
values  of  the  x’s  are  given  by  (4.39)  through  (4.42). 


4.7.  Conclusion 

In  this  chapter  we  have  studied  the  problem  of  simultaneously  designing  communi¬ 
cation  strategies  and  control  policies  for  problems  involving  ARMA  models  of  orders 
higher  than  one.  For  one  of  the  simplest  such  problems,  involving  an  ARMA  model  of 
order  2,  we  have  shown  that  the  optimum  linear  strategy  may  be  outperformed  by  an 
appropriately  chosen  nonlinear  policy.  This  is  done  by  relating  the  problem  involving  the 
second-order  ARMA  model,  to  a  problem  involving  transmission  through  a  Gaussian 
channel  with  noisy  side  information  at  the  decoder.  It  has  further  been  shown  that  over 
the  affine  class,  the  optimal  strategy  consists  of  transmitting  the  innovation  at  each  stage. 


The  solutions  which  are  optimal  over  the  affine  class  have  been  studied  for  the  hard  power 
constraint  version  as  well  as  for  the  soft  power  constraint  version  of  the  general  ARMA 
model. 


u 


8 
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THE  DECENTRALIZED  TWO-PERSON  TEAM  WITH 
MULTIPLE  INFORMATION  CHANNELS 

5.1.  Introduction 

In  this  chapter  we  consider  decentralized,  two-person  stochastic  team  problems, 
where  the  action  of  one  agent  is  transmitted  to  the  other  agent  through  a  number  of  noisy 
channels  simultaneously.  This  is  a  generalized  version  of  the  decentralized  two-person 
team  discussed  in  Chapter  2,  where  only  a  single  communication  channel  was  allowed 
between  the  two  agents. 

In  Section  5.2  we  formalize  the  problems  to  be  analyzed  in  this  chapter.  In  Section 
5.3  we  consider  the  situation  where  the  channel  noises  are  independent  of  the  input  vari¬ 
able  to  be  transmitted,  with  the  observation  of  the  first  agent  being  noise  corrupted  in 
general.  For  this  class  we  show  that  the  optimal  strategies  are  linear  and  may  be  found 
through  a  related  parameter  optimization  problem.  In  Section  5.5  we  analyze  problems 
where  the  channel  noise  is  correlated  with  the  input  variable  to  be  transmitted  and  find 
that  the  strategies  which  are  optimal  over  the  linear  class  may  be  outperformed  by  non¬ 
linear  strategies  (except  for  a  very  restrictive  subclass),  even  when  the  first  agent  observes 
an  uncorrupted  version  of  the  input.  The  concluding  remarks  of  Section  5.4  then  end  this 
chapter. 

5.2.  Problem  Formulation 

We  have  noted  in  Chapter  2  that  if  the  first  agent  observes  a  noise  corrupted  version 
of  the  variable  to  be  transmitted,  and  the  channel  noise  is  also  correlated  with  this  vari¬ 
able,  then  the  linear  strategies  for  the  general  stochastic  team  problem  may  be  outper- 


formed  by  nonlinear  strategies.  In  view  of  this  result,  we  shall  restrict  ourselves  to  the 
following  two  classes  of  problems  with  multiple  information  channels: 

(i)  The  channel  noises  are  all  independent  of  the  input  x,  while  the  first  agent 
observes  a  garbled  version  of  the  input  x. 

’  (ii)  The  first  agent  observes  the  input  x  directly,  and  the  channel  noises  are  allowed 

> 

I 

to  be  correlated  with  the  input  x. 

These  situations  are  depicted  in  Figures  5.1  and  5.2,  respectively.  We  consider  the 

i 

quadratic  cost  functional  J(y0>yi)  where 

i 

J(y0.yi)  =  ElkoUo+SQUoX+su^+SjUjx] 

uo  =  y00)  •  «i  =  y1(0  . 

and  the  following  two  classes  of  problems  are  thus  identified: 

Problem  PI 

Minimize  J(y0.yi) 

>Wi 


where 


Decoder 

*1 

wi  - 

p*d>- 

Encoder  A. 

Figure  5.1.  The  decentralized  two-person  team,  with  channel  noises 
independent  of  each  other  and  of  the  input. 

Decoder 

b  ,x  +w  j 

r— d>— 

,1  . 

Encoder  A, 

b  JC 

6.  x  +w 


Figure  5.2.  The  decentralized  two-person  team  with  channel  noises 
correlated  with  the  input. 


Problem  P2 


Minimize  J(y0.yi) 

yo-yi 


where 


bjX+XjUQ+Wj 

b2x+X2u0+w2 


bnX+Mo+Wn 


u0  =  YoM 

u!  =  yi(y) 

with  x.v.Wj,  .  .  .  ,  wn  being  zero  mean  Gaussian  random  variables,  independent  of  one 

2 

another,  having  variances  denoted  by  a(.),  and  with  A^’s  being  given  positive  constants. 

We  now  turn  to  analyzing  the  above  classes  of  problems. 

5.3.  Instances  with  Optimal  Linear  Solutions 

In  this  section  we  analyze  Problem  PI  formulated  in  the  previous  section  and  show 
that  the  optimal  strategies  are  linear.  This  problem  has  been  studied  in  Bansal  and  Basar 
[1987b]  and  the  analysis  is  given  here  for  completeness.  Towards  obtaining  the  solution 
to  PI  we  define 

m  =  E[x  I  z]  , 

make  observations  (i)  through  (iv)  as  in  the  case  of  the  single  channel  problem  (Section 
2.3  (b))  and  obtain  the  following  equivalent  problem: 


3 


S 

i 


*»■ 


§ 


Minimize  J  (y  0>yi ) 


I 

8 


where 
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2  »1  i 

J'(y'o»y|)  =  E[k0u'0  +s0u'0m+ — (uj-m)  +c] 

4s 


u'o  =  y'0(m) 

ui  =  Vi(y) 


and  c  is  a  constant  independent  of  u'0  and  Uj. 


Let  Jp  denote  the  infimum  of  J  under  the  constraint  E[u02]  =  P2,  i.e.. 


JP  =  inf  JCy'o.yi) 


y,.y,;Ely>PJ 


We  first  have 


Jp  ^  k0P2+  Inf  s0E[u'0m]+  Inf  E(Uj--m)2+c  . 
Ety/l=PJ  Efy/l-p^ 


By  Cauchy-Schwartz  inequality,  we  know  that 


inf  s0E[u'0m]  =  -ls0IPam  . 

E[u'2]=P2 


We  now  consider  the  optimization  problem  Inf  E(uj— m).  Since  m,  y  and  Uj  form  a 

Ety0Jl=P2 


Markov  chain,  we  have 


I{m;u,l  <  I{m;y} 


where  l{a;b}  denotes  the  mutual  information  of  two  random  variables  a  and  b. 


1 


Also,  we  have  the  Inequality 


Using  (5.8)  in  (5.6)  we  get 


Wv 


Hm;y}  <  — logd+P2^— — ) 


(5.9) 


where  we  have  made  use  of  the  expression 


!  11 

H{jMm}  =  —  log(2ire)nII  o2  . 

2  i=i  1 


since  for  fixed  y  ,  the  vector  y  given  m  consists  of  n  independent  Gaussian  random  vari¬ 


ables. 


Using  (5.5)  and  (5.9)  in  (5.4)  we  get 


EKUi-m )  )  >  - 

Ely05l-P2  l+P2^ 


(5.10) 


It  follows  from  (5.2)  (using  (5.3)  and  (5.10))  that 


Jp  >  k0P2  “  I  s0 1  Pam  + - +  c 

(l+A) 

2 

O 

^  Min[k0P2—  I  s0  I  Pam  + - - —  +  c] 

P^°  1+P2\ 

2 

=  k.P*^  —  I  s.  I  P*a_  H - +  c 

o  o  m 

i+p  \ 


(5.11) 


where 


P*  =  Arg  Min[k0P  —  ls0IPom  + 

p 


1+P2X 


-]  >  0  . 


Note  that  P*  necessarily  exists,  since  at  P-0  the  function  to  be  minimized  is  decreasing  and 


as  P-*oo,  J  -*oo,  implying  that  the  search  can  be  confined  to  a  closed  bounded  region  of  R  , 


over  which  a  continuous  function  always  admits  a  minimum.  Taking  derivatives,  we  find 
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that  the  value  P-P*  which  attains  this  minimum  satisfies 


v  ,  1  ,2  2P*o^ 

(2k0P*-ls0lamXP*2  +  -r  =  - 

X  X 


(5.12) 


We  now  have 


Jopt  4  Inf  j(y0.yl)=  Inf  J 


>  k0P  -( ls0l  om)P*  + 


(5.13) 


1+P*2X 


which  gives  us  a  lower  bound  for  the  infimum  of  J.  The  final  task  is  to  note  that  this 
lower  bound  is  tight  and  is  attained  by  linear  strategies.  We  thus  have  Theorem  5.1 
below. 


Theorem  5.1. 

(i)  The  stochastic  team  problem  PI  formulated  in  Section  5.2  admits  an  optimal 
solution  which  is  linear  in  the  observation  variables,  and  is  given  by 

y*Q(z)  =  3*z 

si 

y*,(y)  - - E[x  I  y] 

2s 

Sl  ox2  -  kfi* 

=  - - I - y> 

s  ax2+av2  i=i  a  2[(l/(o  2+ov2))+/3*2\] 

where  3*  is  given  by  the  solution  to  the  following  parameter  optimization  problem: 

s  2  o  4 

3*  =  arg  min  k 0/32(ox2+ov2)  +  s 0/3ox  +  —  • - II - 

8  4s  ( o  2+o  2  X  1  +&2k{  a  x2+  o  v2 ) ) 

a  solution  to  which  may  always  be  found  as  discussed  earlier. 


(ii)  The  optimal  value  of  the  cost  is 


Jopt  =  °x+ay)  +  so$*ax  + - - - 

4s(  a  2+  a  v2)(  l+tf^XC  a  2+a  2)) 


a  a 

X  V 


2  ,  2 
Ox+Ov 


□ 


5.4.  Nonoptimality  of  Linear  Strategies 

In  this  section  we  analyze  problems  where  the  channel  noises  are  correlated  with  the 
input  variable  x  which  is  to  be  transmitted  through  the  channel.  We  show  that  except  for 
a  very  restrictive  subclass  of  such  problems,  the  strategies  which  are  optimal  over  the 
linear  class  may  be  outperformed  by  appropriately  chosen  nonlinear  strategies. 

Since  Xj’s  are  assumed  to  be  nonzero  (otherwise  the  channel  is  redundant  and  may  be 
removed)  the  observation  y  for  Problem  P2  is  equivalent  to  the  following  observation  y\ 

(bj/Xj)  x+u0+(wj/Xj) 

y'=  : 

(bn/Xn)x+uo+(wn/Xn) 

Lemma  5.1.  If  b ll\l  =  b 2/X2  =  •  •  •  =  bn/Xn,  then  the  search  for  optimal  strategies  for  Prob¬ 
lem  P2  may  be  confined  to  the  linear  class. 

Proof.  Under  the  conditions  of  the  lemma  we  may  define 

,  bl 
u  o  =  uo  +  —  x 
^•1 


to  obtain  the  equivalent  stochastic  team  problem  below: 


iwvwTwiniinM 
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Minimize  E[k0u'02  +  s'0u'qX  +  su  2  +  SjUjx] 
y’o-Vi 


where 


u  o  —  y  0(x) 
ui  =  Vi(y) 


and 


y  = 


xiu'o+wi 


V'o+wn 


with 


s  o  ~  s0  2kQ  • 


We  thus  obtain  a  special  case  of  Problem  PI  with  av  =0,  for  which  the  optimality  of 
linear  strategies  has  been  established  in  Section  5.3. 

□ 

We  now  return  to  the  general  Problem  P2  without  the  restriction  of  equality  on  the 
bj/Aj’s,  and  show  that,  for  this  class,  nonlinear  strategies  may  outperform  the  optimal 
linear  strategies. 

In  the  following  we  shall,  for  notational  convenience,  assume  that  As  =  1  for  all  i. 
This  does  not  introduce  any  loss  of  generality  since  a  redefinition  of  the  bj’s  and  the  noise 
variances  yield  the  original  problem. 

It  is  sufficient  to  establish  nonoptimality  of  linear  laws  for  a  special  two-channel 
case,  which  we  study  next. 


s 


8 


5 

b? 


B 

si 


V 

■./ 


§ 


Problem  P2 


where 


Minimize  J2(y0.yi)  —  Efk^+Cuj-x)2] 


u0  =  yo^x) 

Uj  =yj(y) 


yi 

u0+bx+Wj 

y2 

u0— bx+w2 

with  b>0  and  o  2  =  a  2  . 

W1  w2 


Lemma  5.2.  Problem  P2  is  equivalent  to  the  soft  constraint  version  of  the  Gaussian 
Channel  Problem  with  noisy  side  information  at  the  decoder,  studied  in  Appendix  B. 

Proof.  We  need  to  note  that  the  vector  observation  y  is  equivalent  to  observing 


yi+y2 

2 

y-yz 

2 


Wl+W2 


Wj-W2 


and  the  lemma  is  established. 


Since  linear  strategies  are  not  optimal  for  the  problem  witn  noisy  side  information,  in 
view  of  Lemma  5.2  the  linear  strategies  are  not  optimal  for  Problem  P22. 
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We  next  consider  the  more  general  two-channel  problem  with  arbitrary  (but  in  view 
of  Lemma  5.1,  unequal)  bj  and  b2,  but  the  variances  of  wt  and  w2  still  being  equal. 

Problem  P220 

Minimize  J2('yoO'i)  =  Hk0u02  +  s0u,jX  +  (Uj— x)2] 

where 

u0  =  y0Cx) 

Uj  =  yt(y) 

and 


u0+bjX+Wj 

y  = 

u0+b2x+w2 

with  bj  >  b2. 

We  let 


& 

« 


where 


[  UQ+bx+Wj 


y  = 


Uq— bx+w2 


with 


and 


s  o  Sq  kgCbj+bj) . 


The  transformed  problem  may  now  be  viewed  as  one  involving  noisy  side  informa¬ 
tion  at  the  decoder  (cf.  Lemma  5.2),  and  the  nonoptimality  of  linear  strategies  follows. 

5.5.  Conclusion 

In  this  chapter  we  have  studied  decentralized  tv  o-person  stochastic  team  problems 
with  multiple  communication  channels.  For  the  case  when  all  channel  noises  are  indepen¬ 
dent  of  the  input  variable  to  be  transmitted,  we  have  established  the  optimality  of  linear 
strategies.  However,  when  the  channel  noise  is  correlated  with  the  input  variable  we  find 
that  linear  strategies  are  in  general  not  optimal  for  the  case  with  multiple  channels,  even 
when  the  first  agent  observes  an  uncorrupted  version  of  the  variable  to  be  transmitted. 

The  problems  in  this  chapter  may  be  viewed  as  those  involving  encoding  and 
transmitting  over  vector  channels  with  Gaussian  noises  (under  generalized  fidelity  cri¬ 
teria).  where  the  source  output  may  be  distorted  prior  to  encoding  or  the  channel  noises 
may  be  correlated  with  the  source  output. 


The  classical  information  transmission  problem,  viewed  as  a  team  problem,  generally 
assumes  that  one  can  directly  encode  the  variable  to  be  recovered  at  the  receiving  end.  A 
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model  which  allows  distortion  prior  to  transmission  was  first  considered  by  Dobrushin 
and  Tysbakov  [1962].  If  we  suppose  that  the  message  to  be  transmitted  is  a  temperature 
read  by  a  digital  thermometer  or  the  pixel  levels  of  an  image,  then  the  message  would  not 
be  an  exact  copy  of  the  object  of  interest  but  a  noise-bearing  variant  of  it,  and  the  analysis 
in  this  chapter  would  then  be  applicable. 

We  also  find  that  for  the  case  where  the  channel  noise  is  uncorrelated  with  the 
encoder  input,  the  nature  of  the  solution  is  such  that  it  may  be  considered  as  first  extract¬ 
ing  the  message  f rom  its  noisy  version  under  a  mean  square  criterion  and  then  transmit¬ 
ting  this  extracted  message.  This  is  in  accordance  with  the  scheme  reported  for  the  hard 
power  constraint  version  with  n-1  by  Wolf  and  Ziv  [1970],  as  well  as  with  the  "discon¬ 
nection  principle"  introduced  for  finite  alphabets  by  Witsenhausen  [1980]. 

We  should  also  note  that  channels  of  the  form  considered  in  this  chapter  have  been 
called  multipath  channels  in  the  literature  (Ovseevich  and  Pinsker  [1958]).  In  such  chan¬ 
nels,  even  though  there  is  a  single  transmitter,  the  reception  is  as  though  a  number  of 
channels  are  operative  in  parallel.  Pinsker  [1972]  also  gives  expressions  and  estimates  for 
the  quantity  of  information  contained  in  observations  with  respect  to  an  estimated 
parameter  for  a  fixed  and  random  number  of  observations.  For  the  special  case  where 
each  of  the  A.j's  are  assumed  to  be  unity,  our  result  on  maximum  information  between  m 
and  y  (eqn.  (3.9))  corresponds  to  the  expression  derived  in  Pinsker  [1972],  when  the 
number  of  observations  is  fixed,  and  the  variables  are  all  Gaussian.  Pinsker ’s  approach, 
however,  is  based  on  sufficient  statistics,  whereas  here  we  have  employed  known  results 
on  entropy  maximization. 


DECENTRALIZED  MULTIPERSON  TEAMS 

&1.  Introduction 

In  this  chapter  we  formulate  and  analyze  some  decentralized,  multiperson,  stochastic 
team  problems  which  are  generalizations  of  the  two-person  teams  studied  in  Chapter  2.  It 
is  assumed  that  a  (possibly  noisy)  version  of  a  Gaussian  random  variable,  available  at  one 
location,  is  used  to  generate  either  a  single  communication  signal  or  a  set  of  communica¬ 
tion  signals,  which  may  then  be  transmitted  via  noise-corrupted  channels  to  either  a  sin¬ 
gle  location  or  to  a  set  of  locations. 

In  Section  6.2  we  formulate  the  problems  to  be  analyzed  in  this  chapter.  In  Section 
6.3  we  analyze  problems  with  a  single  transmitting  agent,  (synonymously,  encoder)  and 
multiple  receiving  agents  (synonymously,  decoders).  In  Section  6.4  we  analyze  problems 
with  multiple  transmitting  agents  as  well  as  multiple  receiving  agents.  In  Section  6.5  we 
consider  problems  with  multiple  transmitting  agents  and  a  single  receiving  agent.  The 
concluding  remarks  in  Section  6.6  then  end  this  chapter. 

6.2.  Problem  Formulation 

In  this  section  we  formulate  three  classes  of  problems  involving  decentralized  mul¬ 
tiperson  teams,  where  communication  between  agents  is  permitted  only  via  noise- 
corrupted  channels. 

First,  consider  the  situation  depicted  in  Figure  6.1,  where  the  measurement  variable 
u0  is  generated  based  on  a  (possibly  noisy)  observation  of  a  Gaussian  random  variable  x. 
and  is  then  transmitted  over  a  number  of  noisy  channels  to  various  locations.  We  allow 
the  possibility  of  controlling  the  sign  of  transmission  over  each  of  the  channels 


Individually,  and  the  channel  noises  may,  in  general,  be  correlated  with  the  input  variable 
x.  All  agents  are  assumed  to  cooperate  in  minimizing  a  quadratic  cost  functional,  and  we 
formally  have  Problem  PI  below: 


Figure  6.1.  The  decentralized  stochastic  team  with  a  single  transmitting 
agent  and  multiple  receiving  agents. 


Problem  PI 


Minimize 

Yo>')'u'"'1'in;s1"—sn 


Jl(y0;yu . —  ^koUo+roUgX+^pjUjj+qjUjjX] 


Z  =  X  +  V 

Yi  =  si%  +  bix  +  wi  • 

Here  k^pj . Pn  are  given  positive  constants,  r0,  q^s  and  bj’s  are  arbitrary  con¬ 

stants  and  x,  v,  wx, ....  wN  are  independent  Gaussian  random  variables,  with  zero  mean 
and  prescribed  variances.  The  ss's  (i-l,...,N)  are  design  variables,  constrained  to  be  either 
+1  or  -1;  these  control  the  sign  of  transmission  over  each  of  the  channels  individually. 

Note  that  Problem  Pi  involves  the  design  of  only  a  single  communicating  agent,  and 
it  is  only  the  sign  of  transmission  on  different  channels  that  may  be  controlled.  In  case 
we  have  complete  freedom  of  designing  the  communication  variable  for  each  of  the  chan¬ 
nels  individually,  we  obtain  the  situation  depicted  in  Figure  6.2,  which  leads  to  Problem 
P2  below: 


b  +■  w , 


K\ 

V 

Si 


Figure  6.2.  The  decentralized  stochastic  team  with  multiple  transmitting 
as  well  as  multiple  receiving  agents. 


Problem  P2 


N 

Minimize  J2(y01, ....  Yon^ii . =  EtZkiuo!+riuoix+Piuu“kliuiixl 

Yov - Yok'-Yu - -  Yin  i»l 

where 

uoi  =  Voi^ 

un  =  yii^yP 

with 

z  =  x  +  v 
Yi  =  u0j  +  bj  +  W; . 

We  finally  consider  the  situation  shown  in  Figure  6.3,  where  multiple  communica¬ 
tion  agents  are  permitted  and  noise  corrupted  versions  of  all  their  outputs  are  made  avail 
able  at  a  single  location.  This  formally  leads  to  Problem  P3  below. 


Figure  6.3.  The  decentralized  stochastic  team  with  multiple  transmitting 
agents  and  a  single  receiving  agent. 
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Problem  P3 


Minimize  J3(y01 . y0N^^  =  E[£(kiu02i+riu0ix)+p1u12+q1u1x] 


where 


uoi  =  Vof-^ 

ui  =  yi(yi . yN) 


Z  =  X  +  V 


y«  =  %  +  bix  +  w,  • 


The  three  classes  of  problems  identified  in  this  chapter  may  all  be  viewed  as  generali¬ 


zations  of  the  decentralized  two- person  team  problem  studied  in  Chapter  2,  where  it  was 


established  that  linear  strategies  are  optimal  if  either  one  of  the  following  two  conditions 


holds:  (a)  the  channel  noise  is  uncorrelated  with  the  input,  or  (b)  an  uncorrupted  version 


of  x  is  available  at  the  encoder.  Therefore,  in  order  to  study  how  the  results  for  the 


two-agent  problem  carry  over  to  the  multiple-agent  problems,  we  shall  restrict  ourselves 


to  the  following  two  conditions  for  each  of  the  Problems  PI,  P2  and  P3: 


Case  (a):  av  =  0,  b-’s  are  arbitrary,  and 


Case  (b):  av  ^  0,  bs  =  0  for  all  i-1 . N. 


In  the  following  three  sections  we  respectively  analyze  Problems  PI,  P2  and  Pi. 


under  conditions  (a)  and  (  b)  above. 
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63.  The  Problem  with  a  Single  Encoder  and  Multiple  Decoders 

In  this  section  we  study  Problem  PI  identified  in  Section  6.2.  By  completing  squares, 
we  find  that  the  cost  f unction  Jj  may  be  rewritten  as 

N  q.  N  q.2 

Ji(y0;yn . vin)  *  e[ k0u<)2 +r ouox+  T. Pi^uti"*  ” x )2~  Z  ~~ ° x2 1  (6.1) 

i«i  2Pi  i-i  4Pi 

Case  (a):  av2  =  0,  h’s  are  arbitrary. 


If  av  —  0.  we  may  define 


u0i  =  SiU0  +  biX  • 


and  then  consider  the  problem  of  minimizing 


EICUjj  +  —  x)2] 


subject  to 


uii  =  ViiKi  +  WP 


Note  that  for  every  fixed  E[u02]  =  P2,  we  have 


E[u02]  =  P2+b  2ax2+E[2sibiu0x] 


<  P2+b  2ax2+2  I  b;Pax  I 


Since 


Vi  =  %  +  wi 


iV/lVl'f V|  r 


WWA/.:*,:  w..- 


it  follows  losing  (2.10)  and  (2.1 1)  that 


Kxjyj)  <  —  log 
2 


P2+b,2O/+2lbiPo,l+0" 


and  further,  using  (2.9),  we  obtain  that 


* 


2  2 
ax°w( 


E[(unH - x)  ]  >  - • - 

2Pi  4Pj2  (( I P I  + 1  b;  I  ox?+ol) 


Let  J1P  denote  the  minimum  of  Jt  under  the  hard  power  constraint  E[u02]  =  P2.  We 


then  have  (with  K  h  —  ^(q  2/4pi)a  2) 


Jlp^k0P2  +  Inf  E[r0Uox]+  Inf  El^p^u^ - x)2]  +  K 

Eta0J]=PJ  QujhP2  is  {  2Pi 


N  2 

"  9i 


2  2 
cr„  O... 


=  k0P  -  lr0Poxl  +1— ’ - +  K 

i=i4Pi  (  I  PI  +  I  bjl  axf+ol 


„  ,  2  2 
q;  °xaw, 

^  Min  k0P2  -  I  r0Pox  I  +  £  —  • - - - +  K 

p  i=i4Pi  ( I  P I  +  I  b- 1  ov)2+a  2 


l  X  W. 


"  9i2 


2  2 


=  k0P*  -  I  r0P*ax  l+£ - +  K  . 

i=o  4pi  (IP*l  +  lbilax)2+a^ 


w 

fe: 


w 

I/ri.' 


K 

1 
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We  thus  obtain  a  lower  bound  on  the  optimal  cost,  and  the  next  task  is  to  note  that  this 
lower  bound  is  tight  and  is  achieved  by  using  the  linear  policies: 


where 


u*0  =  y*0(x)  =  — sgn  r0 - - 

Ox 

q,  si  I P*  I  +  I  h  I  a  )ox 

«*i«  =  y*iM  —  - - y, 

2Pi  ((  I P*  I  + 1  b;  I  ox)2+a2 ) 


S;  =  sgn  b; . 


(6.10a) 


(6.10b) 


(6.10c) 


We  therefore  have  Theorem  6.1  below. 


Theorem  6.1. 


(i)  The  stochastic  team  problem  PI  with  o  2  =  0  admits  an  optimal  solution  which  is 
linear  in  the  observation  variables,  and  is  given  by  (6.10),  where  P*  is  given  by  the  solu¬ 
tion  to  the  following  parameter  optimization  problem, 


„  2  2 


P*  =  arg  min[k0P  +  r0Pox  +  £ - ] 

p  i=)4Pi  (( I  PI  +  lb;  I  ox)2+o2 


which  always  admits  a  solution. 


(ii)  The  optimal  value  of  the  cost  is 


N  ^ 


2  2 

o„. 


N  2 

_  qi 


J*i  =  k0P*^  -  r0P*ox  +  £ - Z  - a> 

i=i4Pi  (( tP*l  +  lbj  I  ax)2+ou2)  i=i  4Pi 


I 


Case  (b):  bt  =  0  for  all  i,  ar2  is  arbitrary. 

If  bj’s  are  zero  for  all  i,  then  all  channel  noises  are  independent  of  the  input  x.  We 


now  define 


m  4  E(x  I  z) 


and  use  observations  (ii)  through  (iv)  of  Section  (2.3)  to  obtain  the  equivalent  problem 


below: 


N  q.2 

Minimize  E[k0u'02+r0u'0m  +  £  — — (u'u— m)2  +  K] 

y  ii . y  in  ivi  ^Pi 


(6.11) 


u'ii  *  y'xM 

and  K  is  a  constant  independent  of  u'0,u'u, . . . ,  u'1N.  We  thus  obtain  a  special  case  of  the 
problem  discussed  in  part  (a)  of  this  section,  and  the  optimality  of  linear  strategies  fol¬ 
lows.  Note  that  the  S;’s  in  this  case  may  be  chosen  arbitrarily,  since  the  bj’s  are  all  zero; 
hence  in  the  statement  of  the  following  theorem  we  have  taken  Sj  =  +1  for  all  i,  without 
any  loss  of  generality. 


Theorem  6.2. 

(i)  The  stochastic  team  problem  PI,  with  b;  =  0  for  all  i,  admits  an  optimal  solution 


which  is  linear  in  the  observation  variables  and  is  given  by 


u*0  =  \*z 


2pi  \*2(ov2+av2)+c\ 


(6.12a) 


(6.12b) 


where  X*  is  given  by  the  solution  to  the  following  parameter  optimization  problem: 


X*  =  arg  min[koX2C  a  x2+ o  2  a  *  +  £ 


N  q(2 


mi» 


i  4pi  (X2(a  2+av2)+a  2) 


-1 


a  solution  to  which  always  exists  (Remark  2.1). 

(ii)  The  optimal  value  for  the  cost  is 

n  q.2  a 2 )  n  q2 

koX^Ca^+o  2)+roX*ox2  +  Z  7 - “  Z 

i-i  4pi  (X^Co^+o^+o^)  i=i  4pi 


6*4.  The  Problem  with  Multiple  Encoders  and  Multiple  Decoders 

In  this  section  we  analyze  Problem  P2  identified  in  Section  6.2.  First,  by  completing 
squares,  we  find  that  the  cost  functional  J2  may  be  rewritten  as 


N 


q.  q. 

^Toi’  •  •  •  >  Yon'Yu’  *  *  *  ’  =  E[ Z (^iuOi+riu0jX+pj( Uj i+—  x)  -  a  x  ) 

2Pi  4Pi 


(6.13) 


i=l 


Case  (a):  av  =  0,  bj's  are  arbitrary. 

The  analysis  is  similar  to  that  in  Section  2.3.  We  first  define 


u<0i  58  u0i  +  biX 


(6.14) 


to  obtain  the  equivalent  cost  functional 


q. 

J  2(y  oi . y  0N’*y  n»  •  •  •  >  y  in^  =  E(Z^iu  0j+r  jU  0ixH  (u  j j  x)  +K] 

4Pi 


w 

$ 


5 


i 


\j  S 

K\  V 

n 


1 


i  51 


* 


ft 


54  fl 
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where 


u'oi  =  y'otW 

(6.15a) 

ii  =  y'liCu'oi  +  WP 

(6.15b) 

r'j  -  r,  -  2kjbj 

(6.15c) 

and  K  is  a  constant  independent  of  u'01,..tu'0N,u'u . u  in* 

Let  J'2P  denote  the  minimum  of  J'Cy'o.y'j)  under  the  hard  power  constraints 

Etu'oj]  —  Pj2  .  (6.16) 


We  then  have 


N 


J'2P  >  £(k,P,2+  Inf  E[r'jU'0ix]+  Inf  E[-^-(u'irx)2])+K 


i«i 

N 


2‘-“2  tfu^}=P,J  4Pj 


=  £(k,P,2-lrVPiox+^- 


2  °!ol 


■)+K 


i=i 


“Pi  (P2+o2) 


N 


2  2 


2 

>  MinlJ^CkjPj  —  I  r'j  I  Pj<Jx+ - 


o„  o 


(6.17) 


X  w. 


■)+K] 


p^°  i-1 


4Pi  (P  2+a2) 


2  2 
ax  CTw. 


N  q.2 

=  Z(kiP*j2-  I  r'j  I  P*j<rx+— - —  )+K  . 


4P>  (P*j2+o^) 

Noting  that  this  bound  is  tight  and  is  attained  by  linear  policies,  we  have  the  following 


theorem: 


Theorem  63. 


(i)  The  stochastic  team  problem  P2  with  av  =  0  admits  an  optimal  solution  which  is 
linear  in  the  observation  variables  and  is  given  by 


u*oi  =  y  V*)  =  K x 


u*ii  =  vVyP  =  “  ~ 

2  Pi 


CX*i+bi)a> 


Vi 


(6.18a) 


(6.18b) 


where 


X*;  =  ( - b;) 


and  the  PVs  are  found  by  solving  the  following  parameter  optimization  problem: 


(6.19) 


2  9j 

P*j  =  arg  min  (kjPj +r'iPjax+ - 


2  2 
o„. 


4Pi  (P2+a2) 


a  solution  to  which  always  exists  (Remark  2.1). 
(ii)  The  optimal  value  of  the  cost  is 


N 


2  2 


Pi  <7xawi 

J*2=  £(kiP*l2-riP«,„„+— 


0+K 


i=l 


4P|  (p*2+G2) 


(6.20) 


8 


S » 


where 


1 


K  '  L(-Kbi2+r,br—)ox 


i=i 
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Case  (b):  b;  =  0  for  all  i,  aT2  Is  arbitrary. 

If  bj’s  are  all  zero,  then  the  channel  noises  are  all  independent  of  the  input  x.  We 
define 

m  4  E(x  I  z) 

and  use  observations  (ii)  through  Civ)  of  Section  2.3  to  obtain  the  equivalent  problem 
below: 

N  q.2 

Minimize  E[£(kiu'02-friu'0im+— — Cu'^— m)2)  +  K' 


y  . . y  on'?  in  j=i 


4Pi 


with 


u  oi  = 

u'ii  =  V'uCyj) 

and  K'  is  a  constant  independent  of  u'0, . u  on,u  if  •  •  •  *  u  in-  We  thus  obtain  a  special 

case  of  the  problem  discussed  in  part  (a)  of  this  section,  and  the  optimality  of  linear  stra¬ 
tegies  follows. 


Theorem  6.4. 

(i)  The  stochastic  team  problem  P2,  with  b;  =  0  for  all  i,  admits  an  optimal  solution 
which  is  linear  in  the  observation  variable  and  is  given  by 


u*0iU)  =  XV 


u*liCyi)  =  - 


Jh_ 

2Pi 


x*,2(  ox2+o;)+o  2 


(6.21a) 


(6.21b) 


where  A*,  is  given  by  the  solution  to  ihe  following  parameter  optimization  problem: 


,  1,.  2  2  ,  2  -> 

,  q,  ".CA,  ffr+<7w) 

X*i  =  arg  minlkjXj  (<rx  +ctv  )+riXiax  H - ] 

^  4p>  (X2(o  2+a  2)+a  2) 


a  solution  to  which  always  exists  (Remark  2.1). 

(ii)  The  optimal  value  of  the  cost  is 

n  q2  o  2(X*2av2+a 2) 

I(kiX-l2(cr<VavJ)+r1\*iD/+-^ - 

i-i  4p>  (X* 2( a2+a  2)+o  2 ) 


□ 


6^.  The  Problem  with  Multiple  Encoders  and  a  Single  Decoder 

In  this  section  we  study  Problem  P3  identified  in  Section  6.2.  We  find  that  linear 
strategies  are  in  general  nonoptimal  for  this  class  of  problems  even  under  the  restrictions 
that  all  channel  noises  are  independent  of  the  input  and  the  encoder  observes  an  uncor¬ 
rupted  version  of  the  input. 

In  order  to  see  why  linear  strategies  are  not  optimal,  we  consider  the  special  case 
with  N-2  and  b1=b2=ov2=0,  and  impose  the  additional  constraints 

E[u2]  <  P,2  (6.22a) 

E[u2]  <  P22  .  (6.22b) 

We  thus  have  a  problem  of  constructing  real-time  encoding  and  decoding  strategies  for  a 
communication  system  with  hard  power  constraints  where  the  source  dimension  is  one 
and  the  channel  dimension  is  two.  For  this  problem  it  is  known  that  linear  strategies  are 
not  optimal  (Shannon  [1949],  Wozencraft  and  Jacobs  [1965]). 


Our  task  now  is  to  show  that  it  is  possible  to  construct  problems  with  soft  power 
constraints,  for  which  linear  strategies  are  not  optimal.  This  may  be  done  by  an  analysis 


similar  to  that  used  in  Appendix  B.  In  particular,  if  we  let  k1=k2=l(D0.0/(l86.0423)  , 


0^=100.0,  cr  *  —o*  =1.0,  then  the  optimum  strategies  which  are  optimal  over  the  linear 


class  yield  a  cost  of  1.07213,  whereas  nonlinear  strategies  of  the  form 


Uqi  =  y01(x)  =  x 


(6.23a) 


u02  =  Yo2W  =  x  -  sgn  x 


(6.23b) 


CYi+y2+l)/2  if  Yi>0 

(yi+y2-l)/2  if  yj<0 


(6.24) 


yield  a  cost  of  1.06634. 

\ 

We  now  consider  Problem  P3  under  the  additional  restriction  that  the  policies  lie 
within  the  linear  class.  Performance  bounds  and  optimum  linear  coding  schemes  for 
discrete-time  multichannel  communication  systems  have  been  investigated  by  Basar,  San- 
kur  and  Abut  [1980].  They  have  observed  that  when  the  source  dimension  (n)  is  less  than 
the  channel  dimension  (m),  then  the  performance  of  real-time  linear  encoders  is  fairly 
close  to  the  optimum  achievable  performance  found  via  channel  capacity.  This  does  not 
hold  for  the  case  where  n>m,  where  the  performance  is  found  to  improve  significantly  by 
allowing  nonlinearities  in  the  encoding  policies. 

For  stochastic  team  problems  of  the  kind  that  are  of  interest  here,  we  have  n«=l  and 
m-N.  Thus,  in  view  of  the  results  reporied  by  Basar  et  al.  [1980],  it  is  of  interest  to 
study  optimality  over  the  class  of  linear  encoders,  where  the  loss  in  performance  may  be 
offset  by  the  gain  associated  with  avoiding  the  complexity  of  more  general  encoders. 
Optimum  iincar  encoding  and  decoding  strategics  for  the  two-stage  stochastic  team  prob- 
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lem  have  been  studied  in  Basar  [1980],  where  a  rigorous  solution  to  the  general  problem 
has  been  provided.  Here  we  are  interested  in  a  special  case  of  that  general  problem,  i.e., 
the  case  where  the  source  dimension  is  one,  but  with  the  difference  that  there  are  addi¬ 
tional  product  terms  in  the  cost  functional,  the  encoder  input  is  possibly  noise-corrupted, 
and  the  channel  noises  may  be  correlated  with  the  input.  We  now  study  optimality  over 
the  linear  class  for  this  case. 

Under  the  restriction  that  the  encoders  use  linear  policies,  we  have 

u0i  =  A.j(x+v)  (6.25) 

which  implies  that  the  observation  vector  7  is  given  by 

\1(x+v)+b,x+w1 
A.2(x+v)+b2x+w2 

(6.26) 

\N(x+v)+bNx+wN 


The  covariance  of  x  and  y  is  given  by 


Cov(x.y)  = 


(6.27) 


where 
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0 


I 

S  8 


Lyy 


(Xi+b|)2ax2+^iV+< 


(\1+b1XXN+bN)aJ{2+X1XNav2 


(X1+bjXXN+bN)ox2+X1XNav2 


(XN+bN)  ax+XNav+a 


and 


~  IUi+bi)ox . ^N+^N^ax  ^ 


(6.29) 


(6.30) 


We  now  have 


where  £  -  311(1  Z--  are  defined  by  (6.29)  and  (6.30),  with  X:  replaced  by  A.%  where  the 

xy  yy 

X’j’s  are  given  by  the  solution  to  the  following  parameter  optimization  problem: 

N  q2 

(X*x . X*N)  =  arg  min  kiXj2(ax2+av2)+riXiox2+ - e(Xx . XN)]  (6.35) 

<h . *N>  i=i  4Pi 

where  e(Xx . XN)  is  defined  by  (6.32).  (The  above  parameter  optimization  problem 

always  admits  a  solution.) 

(ii)  The  optimal  value  of  the  cost  under  linear  policies  is 
N  q2 

J*j  =  E[£  kjX*j2(ox2+o  v2)+rjX*iox2+ - (a  2-e(X*x . X*N)) . 

i-i  4Pi 

□ 

One  special  case  is  of  particular  interest,  that  with  a  2=0;  bx=b2=..=bN=0, 
rx=r2=..=rN=0.  In  this  case  we  get 


e(Xj , . . . ,  Xj^)  — 


2  2 
2  ax  2  ax 

l+X, - +...+X* 


(6.36) 


which  implies  that  the  optimum  linear  coefficients  (X*x . X*N)  may  be  found  by  solv¬ 

ing  the  following  parameter  optimization  problem: 


Minimize  J(P2 . P^) 

(p,2 . p*) 


where 


J(P2, 


P  2) 


=  i  k,p:+- 


4Pi 


l  +  I(P2/o2) 


1=1 


(6.37 
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and 

Pi2  =  X2ax2  .  (6.38) 

The  optimum  coefficients  from  this  parameter  optimization  problem  may  be  found  by 
applying  Theorem  2.2  of  Basar  [1980]  as  follows.  We  order  the  channels  such  that 

w,  <  •  <  9n0wn  •  (6.39) 

The  derivative  of  J(P2 . P^)  with  respect  to  P 2  is 


1  4pj  n  (6.40) 

d+ZP,2/<’“) 

i=i 

and  clearly  all  first  derivatives  cannot  be  zero  unless  the  product  k;0  2(  is  equal  for  all  i. 

If  this  product  is  indeed  equal  for  all  i,  then  we  may  arbitrarily  choose  one  channel 
over  which  to  transmit  the  information,  and  all  first-order  necessary  conditions  (which 
are  also  sufficient)  are  satisfied. 

On  the  other  hand,  if  the  product  k^  is  not  equal  for  all  i,  we  can  choose  any  chan¬ 
nel  from  the  set  for  which  this  product  is  the  smallest  (this  set  may  be  a  singleton).  For 
the  remaining  channels  with  larger  kjO^  the  first  derivative  is  increasing  at  zero,  implying 

that  zero  is  indeed  optimum. 

Under  the  arrangement  (6.39),  it  is  therefore  sufficient  to  choose  P2  possibly  nonzero 
while  all  P2  for  i-2 . N  may  be  restricted  to  be  zero,  and  we  get 

arg  min  J(P2 . P2)  =  (P* 2,  . .  .  ,  P*N2) 

(Pi . Pn> 


where 


V*J 

'•‘.I 
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*% 
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'i 

J.i 
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3; 

I 
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's 


if 

i*.1 
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P*2  «  sqrt(Max{0,- 


°x°w, 


1  2 

- O  }) 

Vi  1 


(6.41) 


P*i  =0  for  all  1=2 . N 


We  therefore  have  the  following  corollary  to  Theorem  6.5. 


Corollary  6.1. 

Consider  Problem  P3,  under  the  additional  restrictions  that 

2 

ov=0;  bj=b2=..=bN=0;  rj=r2=..=rN=0 . 

If  we  assume  that  the  channels  are  ordered  such  that  (6.39)  holds,  then  the  optimal  linear 


strategies  are  given  by 


u*oi  =  yVx)  =  K*\x 

u*0i  =  -  0  for  a11  i==2 . N 


u*,  =  y*j(y)  - - 


*Vx 


2p'  **,V+< 


where 


2  1  2 
x*,2  =  —  p*,! , 

O  " 


and  P*,  is  defined  by  (6.41 ). 

(ii)  The  optimal  value  of  the  cost  is 
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t.P*!2  +  — 


2  2 


4Pi  (P *?+ol) 


(S' 
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&6.  Conclusion 


In  this  chapter  we  have  formulated  and  analyzed  three  classes  of  decentralized,  mul¬ 
tiperson,  stochastic  team  problems,  where  communication  between  agents  is  permitted 
only  via  noise-corrupted  channels. 

For  problems  involving  a  single  transmitting  agent  and  multiple  receiving  agents  we 
have  shown  that  the  optimum  strategies  are  linear  when  either  the  encoder  observes  an 
uncorrupted  version  of  the  variable  to  be  transmitted,  or  when  all  channel  noises  are 
independent  of  the  input. 

For  problems  involving  multiple  transmitting  agents  and  multiple  receiving  agents 
we  again  find  that  in  case  the  encoders  observe  an  uncorrupted  version  of  the  variable  to 
be  transmitted,  or  if  all  channel  noises  are  independent  of  the  input,  then  the  optimum 
strategies  are  linear. 

However,  for  the  simplest  classes  of  problems  involving  multiple  transmitting  agents 
and  a  single  receiving  agent,  we  find  that  the  strategies  which  are  optimal  over  the  linear 
class  may  be  outperformed  by  appropriately  chosen  nonlinear  policies,  and  for  this  case 
we  have  provided  strategies  which  are  optimal  within  the  linear  class. 
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CHAPTER  7 

PROBLEMS  WITH  INCOMPLETE  CHANNEL  DESCRIPTION 


cv 


7.1.  Introduction 

In  this  chapter,  we  expand  on  the  framework  developed  so  far,  by  allowing  incom¬ 
plete  statistical  description  of  the  channel  used  to  transmit  measurements  between  the 
decentralized  agents.  We  still  operate  under  nonclassical  information  patterns  and  con¬ 
sider  a  number  of  cases  depending  on  whether  there  are  “hard”  power  constraints  or 
“soft”  power  constraints  on  some  of  the  decision  variables  and/or  soft  costs  on  communi¬ 
cation.  We  obtain  minimax  rules  in  all  cases,  some  being  saddle  points  and  others  not,  the 
techniques  of  derivation  being  very  much  case  dependent.  These  are  all  important  proto¬ 
type  problems  which  could  be  considered  essential  building  blocks  in  multistage,  distri¬ 
buted  decision  making  under  nonclassical  information  and  with  partial  statistical  descrip¬ 
tion. 

In  particular,  we  assume  that  the  variable  transmitted  through  the  channel  is  cor¬ 
rupted  not  only  by  an  independent  Gaussian  noise  of  given  variance,  but  also  by  an  unk¬ 
nown  channel  noise  which  is  only  known  to  satisfy  certain  power  constraints.  Further, 
this  unknown  noise  is  allowed  to  be  correlated  with  either  the  input  or  the  output  of  the 
encoder.  (Recall  that  in  Chapter  2  we  had  allowed  part  of  the  channel  noise  to  vary 
Linearly  with  the  input). 

We  seek  optimal  solutions  under  a  worst  case  analysis.  We  may  therefore  consider 
the  unknown  channel  noise  as  being  controlled  by  an  adversary  or  “jammer,”  who  intelli¬ 
gently  uses  the  knowledge  of  either  the  input  or  the  output  of  the  channel  to  design  a 


$ 


9 


jamming  strategy.  Thus  the  problems  may  be  viewed  as  those  representing  extended  ver¬ 
sions  of  the  standard  Gaussian  Test  Channel  (Gallager  [1968]),  further  including  an  intel¬ 
ligent  jammer. 

The  organization  of  this  chapter  is  as  follows.  In  Section  7.2  we  identify  the  prob¬ 
lems  to  be  analyzed  in  the  sequel.  In  particular,  three  kinds  of  communication  systems 
are  identified,  each  of  which  is  subsequently  analyzed  under  a  variety  of  fidelity  criteria, 
which  are  also  defined  in  Section  7.2.  The  three  types  of  communication  systems  are 
analyzed  in  Sections  7.3,  7.4  and  7.5.  The  concluding  remarks  in  Section  7.6  then  end  this 
chapter. 

In  what  follows,  we  shall  refer  to  the  problems  studied  in  this  chapter  alternatively 
as  zero-sum  games,  since  the  jammer  wishes  to  maximize  the  same  criterion  which  the 
encoder-decoder  pair  is  trying  to  minimize.  A  general  discussion  on  zero  sum  games  may 
be  found  in  Basar  and  Olsder  [1982]. 

12,.  Problem  Description 

7.2.1.  Channel  description 

Following  the  formulation  of  Basar  [1983],  consider  the  communication  systems  dep¬ 
icted  in  Figures  7.1,  7.2  and  7.3  which  represent  extended  versions  of  the  standard  Gaus¬ 
sian  Test  Channel,  (Gallager  [1968]),  and  include  an  intelligent  jammer  who  has  access 
either  to  the  input  or  to  the  output  of  the  encoder.  The  input  to  the  encoder  is  a  Gaussian 
random  variable  with  zero  mean  and  unit  variance,  denoted  u  ~  N(0,1).  The  transmitter 
encodes  the  input  signal  u  into  a  variable  y  =  y(u)  with  the  encoding  policy  y  being  an 
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element  of  the  space  I"e  of  random  mappings  or  an  element  of  the  space  of  determinis¬ 
tic  mappings.  The  encoder  is  further  restricted  to  satisfy  certain  power  constraints  as  to 
be  elucidated  in  the  sequel.  When  the  encoder  map  is  allowed  to  be  probabilistic  we  refer 
to  the  problem  as  a  game  of  Type  1,  in  which  case  we  also  allow  for  a  side  channel  that 
permits  the  transmission  of  information  regarding  the  structure  of  the  probabilistic 
encoder  to  the  receiver  (Figure  7.1).  When  the  encoder  map  is  necessarily  deterministic  it 
is  a  game  of  Type  2,  for  which  the  side  channel  is  superfluous  (Figure  7.2). 

In  games  of  Types  1  and  2  the  jammer  is  allowed  to  observe  the  input  u.  The  jammer 
then  feeds  back  into  the  channel  a  random  variable  v  =  /3(u)  so  that  the  input  to  the 
decoder  is 

z  =  y(u)  +  j8(u)  +  w  . 

The  jammers  policy  is  an  element  in  the  space  T-  of  random  mappings  also  restricted  to 
satisf y  certain  power  constraints. 

At  the  receiver,  the  decoder  8(z)  is  chosen  so  as  to  cooperate  with  the  encoder  in 
minimizing  a  (generalized)  quadratic  distortion  measure  J(y,8,/3).  Here  the  noise  w  is 
assumed  to  be  Gaussian  with  mean  zero,  variance  o^,  and  is  independent  of  u. 

Figure  7.3  represents  the  situation  when  the  jammer  has  access  to  the  output  of  the 
encoder  instead  of  u.  This  situation  shall  be  referred  to  as  a  game  of  Type  3.  Games  of 
Type  3  are  investigated  in  Bansal  and  Basar  [1989]  under  a  variety  of  fidelity  criteria 
which  we  next  describe. 

7.2.2.  Fidelity  criteria 

The  general  communication  game  addresses  the  reproduction  of  the  input  u.  with 
high  fidelity,  at  the  output  of  the  decoder.  This  corresponds  to  minimizing  the  mean 


square  error  E[(8(z)  —  u)2}.  In  order  that  this  minimization  be  well  defined  we  restrict 
ourselves  to  using  encoding  and  jamming  policies  which  satisfy  E[y2(u)]  <  P02  and 
E[v  ]  <  Pj .  These  are  the  hard  power  constraints  generally  assumed  in  the  communica¬ 
tion  literature,  which  lead  to  criterion  Ct  below. 

Criterion  Ct: 

JiCy.8,/3)  =  E{(8(z)  -  u)2}  (7.1a) 

and 

E[y2(u)]  <  P02  ,  E[v2]  <  P/  .  (7.1b) 

This  is  precisely  the  criterion  adopted  in  Basar  [1983]  and  Basar  and  Wu  [1985], 
under  which  a  complete  characterization  of  minimax  encoder  decoder  policies  for  games  of 
Types  1  and  2  has  been  provided  in  Basar  and  Wu  [1985].  We  introduce  here  some  classes 
of  problems  which  have  heretofore  received  less  attention.  These  are  problems  where  the 
power  levels  are  not  fixed  a  priori,  but  are  determined  as  a  result  of  the  underlying  optim¬ 
ization  problem.  This  is  particularly  important  in  situations  where  it  may  be  possible,  for 
instance,  to  transmit  a  larger  power  at  the  encoder  (at  some  cost)  in  order  to  further 
decrease  the  mean  square  error  at  the  decoder.  Mathematically  this  may  be  represented  as 
a  power  constraint  which  is  “implied”  or  "soft”  and  appears  as  an  additional  term  in  the 
cost  functional.  Based  on  the  consideration  of  tradeoffs  between  power  used  and  the  mag¬ 
nitude  of  the  mean  square  error  term,  we  have  criteria  C2,  C3  and  C4  below  where 
coefficient  terms  k0  and  a  are  positive  scalars  in  all  cases. 
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Criterion  C2: 


Criterion  C, 


Criterion  C. 


J2(y,8,/3)  =  Ef^/y^u)  +  (8(z)  -  u)2  -  av2} 


J3(y,8,/3)  =  E{(8(z)  —  u)2  —  av2} 
E[y2(u)]  <  P02 

-T4(y.8,/3)  =  E{  k0y2(u)  -I-  (8(z)  —  u)2} 
E[v2(u)]  ^  P,2  . 


(7.2) 


(7.3a) 

(7.3b) 


(7.4a) 

(7.4b) 


Under  the  different  criteria  above,  the  spaces  in  which  the  policies  need  to  be  res- 
tricted  are  characterized  by  using  a  superscript.  For  example,  denotes  the  class  of 
deterministic  encoder  policies  which  are  permissible  when  criterion  Ck  is  under  considera- 


7.2.3.  Solution  concepts 

We  are  faced  with  multiperson  decision  problems  with  conflicting  objectives.  We 
thus  seek  to  obtain  worst  case  solutions  by  minimax  and  maximin  approaches,  and 
further,  saddle-point  solutions  if  they  exist. 

Under  the  minimax  approach  the  encoder-decoder  pair  is  assumed  to  be  careful  and 
defensive,  wishing  to  protect  against  any  irrational  behavior  of  the  jammer.  Thus,  under 
the  minimax  criterion,  we  evaluate  the  upper  value  J  of  the  zero  sum  game  with  kernel  J, 

J  =  J(y*,8*,/3*(r  5,j)  =  min  max  J(y,8,/3) 

yS  S  (7.5) 

=  min  J(y,8,/3*(y  6)) 
y.s 


where 


tt.  1?,-  tiL  ij.  iV.  'tH  .  *>L^i(  fill i1.!* t.>'t.l*t J  hi' #  I* I . lH'  4tI' »*i «*r  krt*|^ I*  1^1  tV 1^1*  !«*  l«*  L| 


/3*(  s)  =  arg  max  J(y, 8,3) 


assuming  that  such  a  solution  exists.  The  triple  J(y*,S*, 3*(y,,s.))  defined  by  (7.5)  and  (7.6) 
above  is  called  the  minimax  solution  for  the  corresponding  game. 


To  obtain  the  maximin  solution,  we  consider  instead  the  lower  value  of  the  zero  sum 


game: 


±=  J(y*fl,.8*fl,./3*)  =  max  min  J(y,Sj3) 

*  (y.S) 

=  max  Ky*0,8*B,&) 
a 


where  the  pair  (y^.S^)  is  determined  by 

(yV8V  =  arg  min  J(y,8,3)  •  (7.8) 

(y,6) 

The  triple  (y*0,.8*e„3*)  defined  by  (7.7)  and  (7.8)  above  is  the  maximin  solution  for  the 
corresponding  game. 

A  saddle-point  solution  (y*,8*  3*)  will  exist  if  and  only  if 

2=j  =  J(y  *,5*,3*).  (7.9) 

A  detailed  study  of  the  concepts  of  mimmax  and  maximin  strategies  and  saddle-point 
solutions  may  be  found  in  Basar  and  Olsder  [1982]. 


7.2.4.  Notation 

We  shall  use  a  double  subscript  notation  and  refer  to  a  game  as  Game  Gik.  with 
i€{  1.2,31  and  k € { ! ,2.3,4}.  The  first  subscript  i  refers  to  the  game  type  (Section  7.2.1) 
while  the  second  subscript  k  indicates  that  fidelity  criterion  Ck  is  under  consideration. 
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with  the  strategy  spaces  being  or  rek,  Trk  and  T*  for  the  encoder,  decoder  and  jammer, 
respectively.  Thus  for  i-3  the  jammer  has  access  to  the  output  of  the  encoder,  whereas 
for  i-1  or  2  the  jammer  taps  the  input  u.  Further,  for  i-1  the  encoder  is  allowed  to  use 
probabilistic  mappings  whereas  for  i-2,  the  encoder  is  necessarily  deterministic. 

7.2.5.  Contributions  of  the  chapter 

The  existence  of  a  saddle-point  solution  for  game  G31  is  proved  in  Basar  [1983]  in  a 
more  general  framework  where  it  is  shown  that  the  parameter  space  can  be  partitioned 
into  regions  with  the  saddle  point  depending  structurally  upon  the  region  of  operation. 
Games  G;1  for  i-1, 2  are  considered  in  Basar  and  Wu  [1985]  where  the  existence  of  a 
saddle-point  solution  for  Gu  is  established  and  a  complete  characterization  of  minimax 
and  maximin  strategies  for  G21  is  provided. 

In  this  chapter  we  study  the  remaining  nine  games  Gik,  i€{  1,2,3}  and  k€{2,3,4}.  In 
Section  7.2  we  provide  saddle-point  solutions  for  games  G12,  G13  and  G14.  In  Section  7.4 
we  analyze  games  G32,  G33  and  G34.  For  games  G22,  G23  and  G24  saddle-point  solutions  do 
not  exist;  we  provide  both  maximin  and  minimax  solutions  for  these  games  in  Section  7.5. 

13.  Saddle-Point  Solutions  for  Games  G12,  G13  and  G14 

Consider  the  problem  described  in  Section  7.1,  with  the  encoder  allowed  to  use  pro¬ 
babilistic  mappings  (Figure  7.1 ),  under  Criteria  C2,  C3  and  C4,  respectively.  The  solution 
is  provided  in  Theorem  7.1  afler  introducing  some  notation. 
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1,1  *,» 


Preliminary  Notation  for  Theorem  7.1 


We  introduce  scalar  parameters  P*0  and  P*j  by  dividing  the  parameter  space  into  dis¬ 


tinct  regions  as  follows: 


a.  Game  G. 


Define  Regions  R12,  R12  and  R12  by 


n 1  1  ^  2 

R.,  :  —  <  o„. 


(k0+a) 


<"  2  ^  * 
—  $  o  <  — 

2  w  w 
Ko 


n  3  2  .  ^0 

RU  :  °w  <  - - 


(kg-t-O-) 


Now  define  P*0  and  P*j  by 


p*o  = 


(  z  >v 


(k0+a) 


in  R12  and  Rj2, 


P*  = 


z  y* 

O 


( k0+o) 


in  R,32 


(7.10a) 


(7.10b) 


(7.10c) 


(7.11a) 


(7.11b) 


: 


b.  Game  G13 


Define  Regions  R13  and  R  23  by 


R13  :  a  >  P02/(P02  +  ol? 


R,3  :  a  <  P02/(P02  +  a  2)2  . 
Now  define  P*0  and  P*j  by 


P*o  =  po 


in  R|3  and  R^3 


(7.12a) 

(7.12b) 


(7.13a) 


in  R,3 


P*,= 


1  (P0/a*)-(P02+a2)  inR23 


(7.13b) 


c.  Game  GI4 

Define  Regions  R14  and  RI4  by 

R,4  :  k0  ^  (P/  +  olT1 

R,l4:k0<(p/  +  ov;.r1. 

Now  define  P*0  and  P*j  by 


(7.14a) 

(7.14b) 


P*0  = 


( ( ( P,2+02)*/  k0Vz)-(  P/+0  w))’ 


(7.15a) 


P*i  =  P, 


in  R14  and  R24  . 


(7.15b) 


Theorem  7.1:  Consider  the  games  G]2.  G, ,  and  G]4  described  in  Section  1  For  each  of 
these  games  there  exists  a  saddle-point  solution  (y*  8*0*)  given  as  follows: 


144 


(P*0u,(P*0/(P*o2+^w+p*/^z) 


(y*  8*)  = 


(-P*0u,(-P*0/CP*02+a>P*/))z) 


w.p.  0.5 

w.p.  0.5 


C7.16) 


3*(u)  =  n  (7.17) 

where  r>  is  a  zero  mean  Gaussian  random  variable  with  variance  P*/,  which  is  independent 
of  u  and  w. 


Proof:  We  need  to  prove  that  the  solutions  given  above  satisfy  the  saddle-point  inequali¬ 
ties: 

J(y*,8*,3)  <  J(y*,8*,3*)  <  J(y, 8.3*)  (7.18) 

for  all  permissible  triplets  (y,8,3)  and  under  the  stipulation  that  the  side  channel  is  used 
io  carry  structural  information  concerning  the  probabilistic  encoder  mappings.  The  proof 
is  provided  by  establishing  separately  the  validity  of  the  right-hand  side  (RHS)  and  left- 
hand  side  (LHS)  inequalities  of  (18)  under  each  criterion  for  each  of  the  regions  in  con¬ 
sideration. 

(a)  Game  G1Z 

Region 

(i)  The  RHS  inequality 

The  problem  faced  by  the  encoder-decoder  pair,  with  the  jammer's  policy  fixed  as 
indicated  is 

min  E[k0y2(u)  +  (8(z)— u)2] 

y.s 
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where  z  =  y(u)  +  w. 


For  each  fixed  power  level  (E[y2(u)]  =  P02)  the  problem  above  is  the  standard  Gaus¬ 
sian  test  channel  for  which  the  optimal  solution  is  known  to  be  linear.  The  optimal  power 


levels  may  be  found  by  solving  the  parameter  optimization  problem: 

min  [k^Q  +  <r  2/( P02  +  o*)] 

p« 

which  gives  P*02  =  0  as  in  the  statement  of  the  theorem. 

(ii)  The  LHS  inequality 

With  the  encoding-decoding  policy  fixed  at  zero,  the  problem  faced  by  the  jammer  is 

max  (— a/32(u)  +1) 

/3(u) 

for  which  the  optimal  choice  is  /3*(u)  =  0. 

Region  R*2 

(i)  The  RHS  inequality 

As  in  the  case  of  Region  R^,  we  find  that  the  optimal  encoding-decoding  policy  is 
linear  and  the  linear  encoding  coefficient  may  be  found  by  solving  the  parameter  optimiza¬ 
tion  problem 

min  [k0P02  +  o^/(P02  +  o^)] 

p0 

which  gives  P*02  =  (ow/ k,/2)  —  a*  as  in  the  statement  of  the  theorem. 

(iij  The  LHS  inequality 

With  the  encoding-decoding  policy  fixed  as  given,  the  cost  to  be  maximized  by  the 
jammer  is  obtained  by  unconditioning  the  conditional  value  of  the  cost  incurred  for  each 


« <*.  ft  * . a, * .  v ,  *  •  t ' : . v  j.1'* 


;  ri.t'l  I'.  I'i  ■*>  ’  t,|  ».»  ft  k 


realization  of  the  equiprobable  random  variable  which  decides  the  sign  of  the  linear 
encoding.  In  the  expression  thus  obtained  for  the  cost  to  be  maximized,  the  jammer's  pol¬ 
icy  only  enters  as  a  term  containing  E[/32(u)]  with  the  coefficient 


(((ow/k0%)  -  <7  2)k0/a  2)  -  a  . 


This  coefficient  is  nonpositive  in  R22,  implying  that  /3(u)  =  0  is  the  jammer’s  optimal  pol- 


Region  R*2 


(i)  The  RHS  inequality 

When  the  jammer  uses  independent  Gaussian  noise  with  the  indicated  variance,  the 
problem  f aced  by  the  encoder-decoder  pair  is 

min  E[k0y2(u)  +  (S(z)—u)2  +  K'] 

y.S 


where  z  =  y(u)+w+T)  and  K'  is  independent  of  the  encoding-decoding  policy.  For  each 
fixed  power  level,  E[y2(u)]  =  P02,  the  problem  is  the  standard  Gaussian  test  channel  prob¬ 
lem,  and  thus  the  coefficient  of  the  optimal  linear  solution  may  be  found  by  solving  the 
parameter  optimization  problem 


min  [k0P0  +  ( k0/ ( k0+a )  )/ ( P0  +( kQ/ ( k0+<* r ) )] 
p„ 


which  gives  P*0  as  in  the  statement  of  the  theorem. 


(ii)  The  LHS  inequality 

When  the  encoder-decoder  policy  is  fixed  as  indicated,  the  cost  function  to  be  maxim¬ 
ized  by  the  jammer  is  obtained  by  unconditioning  the  conditional  value  of  the  cost  which 
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gives  an  expression  independent  of  j8,  and  thus  the  jamming  policy  may  be  chosen  as  in 
the  statement  of  the  theorem. 

(b)  CameG13 

Region  R^3 

(i)  The  RHS  inequality 

With  the  jammer’s  policy  fixed  at  zero,  the  encoder-decoder  pair  faces  the  standard 
Gaussian  test  channel  problem  for  which  the  optimal  linear  solution  is  as  indicated  in  the 
statement  of  the  theorem. 

(ii)  The  LHS  inequality 

When  the  encoder-decoder  policy  is  fixed  as  indicated,  the  cost  function  to  be  maxim¬ 
ized  by  the  jammer  is  obtained  by  unconditioning  the  conditional  value  of  the  cost  which 
gives 

E[(-a  +  Po2/(Po2+a^.)2)02(u)+K"] 

where  K"  is  independent  of  /3.  In  the  region  under  consideration  the  coefficient  of  E[|32(u)] 
is  negative  implying  that  /3(u)  =  0  is  an  optimum. 

Region 

(i)  The  RHS  inequality 

With  the  jammer’s  policy  fixed  as  given,  the  encoder-decoder  pair  faces  the  standard 
Gaussian  test  channel  problem,  and  the  solution  is  as  given. 


C'» 

ii. 


(ii)  The  LHS  inequality 

With  the  encoding-decoding  policy  fixed  as  indicated,  the  cost  function  to  be  maxim¬ 
ized  by  the  jammer,  obtained  by  unconditioning  the  cost,  becomes  independent  of  j3(u) 
which  may,  therefore,  be  chosen  as  in  the  statement  of  the  theorem. 


(c)  Game  G, 


Region  R14 


(i)  The  RHS  inequality 

With  the  jamming  policy  fixed  as  Gaussian  noise,  the  problem  faced  by  the  encoder- 


decoder  pair  is 


min  E[k0y  (u)  4-  (8(z)— u)2] 
y.S 


with  z  =  y(u)+w+7)- 


The  coefficient  of  the  optimal  linear  solution  to  the  above  problem  may  be  found  by 
solving  the  parameter  optimization  problem 

2  ,  r,  2 

2  °*+PJ 

min  k0P0  + - 

P°  po2+Pj2+<^ 

for  which  P*Q2  =  0  in  the  region  under  consideration. 

(ii)  The  LHS  inequality 

With  the  encoder-decoder  policy  fixed  at  zero,  the  cost  function  is  independent  of 
0(  u)  which  may  be  chosen  to  be  Gaussian  noise  as  in  the  statement  of  the  theorem. 


ji. 


Region  Rt2 


(i)  The  RHS  inequality 

The  problem  faced  by  the  encoder-decoder  pair,  with  the  jammer’s  policy  fixed  as 
Gaussian  noise  is 

min  E[k0y2(u)  +  (8(z)— u)2] 

y.s 

where  z  =  ■y(u)+w+,n. 

The  coefficients  for  the  optimal  linear  solution  to  the  above  problem  may  be  obtained 
by  solving  the  parameter  optimization  problem 

min  k0P02  +  (a  2+PJ2)/(P02+PJ2+o  2) 

po 

which  gives  the  optimal  encoding-decoding  policies  as  in  the  statement  of  the  theorem. 

(iij  The  LHS  inequality 

With  the  encoding-decoding  policy  fixed  as  indicated,  the  cost  to  be  maximized  by  the 
jammer,  obtained  by  unconditioning,  increases  linearly  with  E[/32(u)].  Thus  the  jammer 
may  choose  any  random  variable  with  second  moment  equal  to  Pj2;  i.e.,  $(u)  may  be 
chosen  to  be  independent  Gaussian  noise  as  in  the  statement  of  the  theorem. 
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7.4.  Analysis  of  Games  ^32»  ®33  and  G34 


7.4.1.  Some  useful  results 


The  most  relevant  paper  to  this  section  is  Basar  [1983],  where  a  game  of  Type  3  with 


fidelity  Criterion  Cl  has  been  studied.  Here  we  reproduce  the  main  theorem  of  that  paper 


(in  the  current  less  general  framework)  for  future  reference.  First  note  that  if  P0  <  Pj 


the  solution  is  trivial  since  the  jammer  can  cancel  out  the  signal  component.  If,  however, 


P0  >  Pj,  then  it  is  convenient  to  partition  the  parameter  space  into  two  regions 


Rl:  P0  >  Pj  and  P0Pj  -  Pj2  <  a  2 


R2:  P0  >  Pj  and  P0P:  -  Pj2  ^  a  2 


and  to  introduce 


=  -  *VP0 


=  1  -(Pj'  +  o2)2/P02Pj2 


Lemma  7.1.  Game  G31  admits  two  saddle-point  solutions  (y*,8*,/z*)  and  (— y 8*,p*), 


where 


(i)  y*(u)  =  P0u 


(ii)  /jl*  is  the  Gaussian  probability  measure  associated  with  the  random  variable 


in  Rl 


v  =  /3*(y)  = 


\*(  1— t*)'Ay  +  -n*  ‘n  R2 


where  n*  ~  (O.t’Pj  ) 
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(iil)  8*  is  the  Bayes  estimator  for  u  under  the  least  favorable  distribution  (i*,  com¬ 
puted  as 


(P0(l+\*)/[P/(l+\*)2+o  2])  z 


in  R1 


«z)  = 


(1/P0)z 


in  R2 


Proof.  See  Basar  [1983]. 


In  order  to  establish  similar  results  under  Criteria  C2,  C3  and  C4,  we  shall  need  the 
following  lemmata: 

Lemma  7.2.  Consider  the  stochastic  team  problem  with  nonclassical  information  (which 
can  be  obtained  from  C2  by  setting  /3(0  =  0) 

min  E{k0y2(u)  +  (8(z)  -  u)2} 

y.S 

where  z  =  y(u)+w.  The  optimal  solution  to  this  problem  is  linear,  i.e., 

y*(u)  =  P*0u 

and 

8*(z)  =  E[u  I  P*0u+w]  =  [P*o/(P02+a  2)]  z  , 

where 

P*0  4  arg  min  { k0P2  +  a^,/(P2+o^)[  . 

p 

Proof.  See  Bansal  and  Basar  [1987a]. 

Lemma  7.3.  Consider  the  stochastic  team  decision  problem  with  nonclassical  information 
(which  can  be  obtained  from  C3  by  setting  j3(-)  =  0) 
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min  E[(8(z)  — u)z} 


2  2 

where  z  —  y(u)+w  and  E[y  Cu)]  ^  P  .  The  optimal  solution  is  linear  and 


y*(u)  =  Pu  , 


8*(z)  =  [P/(P2+oi)]  z  . 


Proof.  The  problem  may  be  viewed  as  the  standard  Gaussian  Test  Channel  problem, 


from  which  the  optimality  of  the  linear  solution  follows.  Alternatively,  the  proof  fol¬ 


lows  in  a  straightforward  fashion  using  a  contradiction  on  Lemma  7.2. 


We  note  here  that  because  of  the  complex  structure  of  the  solution  to  the  hard  con¬ 


straint  problem  (discussed  in  Basar  [1983]),  the  solution  given  in  Lemma  7.1  cannot 


readily  be  used  to  obtain  the  solution  to  the  soft  constraint  versions.  In  fact,  it  turns  out 


that,  for  each  of  the  soft  constraint  versions,  the  saddle-point  property  has  to  be  esta¬ 


blished  individually. 


7.4.2.  Solutions  to  the  minimax  problem  under  the  three  criteria 


(a)  Game  G, 


Preliminary  notation  for  Theorem  7.2. 


We  decompose  the  parameter  space  into  three  regions  Rl,  R2  and  R3: 


R3'2:  a  <  k0 


R322:  a  >  k0  and  cxk0  -  k02  <  a 3a  2 


R32:  «  >  k0  and  ork0  -  kQ2  ^  a3a^  . 
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We  next  introduce  the  scalar  parameters  P*0  and  X*  by 


P*0  =  (Max|0,aw((ko-1— a-1)'4— ow)/((l— k0/o')2)})’A  in  R' 


1  :•/ 


X*  = 


-k0/a  in  R322  and  R32 


Theorem  7.2.  Game  G32  admits  two  saddle-point  solutions  and  C— y*  — 


where 


(i)  y*(u)  =  P*0u 

(ii)  fi*  is  the  Gaussian  probability  measure  associated  with  the  random  variable 


in  R32  and  R322 


v  =  /3*(y)  = 


X*y  +  n*  in  r332 


where  n*  is  an  independent  Gaussian  random  variable,  i.e., 


T)*  ~  N(0,a^.)  =  N(0,ko/c*2  -  k  2/a3  -  o*) 


(iii)  5*  is  the  Bayes  estimator  for  u  under  the  least  favorable  distribution  /a*,  com¬ 


puted  as 
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(P*0(l+X*)/(P*2(l+X*)2+o^))  z  in  R32  and  R322 


8*(z)  =  p*z  = 


(1/P*  )  z 


in  R32  . 


Proof .  We  shall  establish  separately  the  validity  of  the  right-hand  side  (RHS)  and  left- 


hand  side  (LHS)  inequalities  of  (7.18)  in  each  of  the  Regions  R32,  R32  and  R32 


Region  R32 

(i)  The  RHS  inequality: 

With  j3(y)  =  /3*(y)  =  — y,  the  problem  faced  by  the  encoder-decoder  pair  is 

min  E{(k0— t*)y2(u)+l} 
y.S 

which  implies  y*(u)  =  0  and  8*( z)  =  E[u  I  w]  =  0 

(ii)  The  LHS  inequality: 

With  y*  =  8*  =  0,  the  problem  faced  by  the  jammer  is 

max  E{  1  —  cd32(y)} 

and  since  y-0  a.s.,  any  choice  of  3  with  E[02(y)]=O  attains  the  maximum  (which  is  1 ),  one 
such  policy  being  8(y)  =  — y. 


Region  R32 

(i)  The  RHS  inequality: 

With  the  jammer's  policy  fixed  as  3*(y )  =  (— k0/a)y,  the  problem  faced  by  the  encoder- 
decoder  pair  is 

min  E[(k0— k,2/c*)y2(u)  +  (8(z)— u)2] 

y.S 


***  **.  I*.  r  'r 
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with  z  =  (1—  k0/a>y(u)  +  w. 

Defining  y'(u)  —  (1— k0/a>y(u),  the  problem  may  be  rewritten  as 

min  E[k'(y'(vO)2  +  (8(z)— u)2] 

Y.6 

where  z  =  y'(u)+w.  It  follows  from  Lemma  7.2  that  the  optimal  encoder-decoder  policies 
are  linear  in  the  observations,  and  may  be  obtained  by  minimizing  over  P 2  >  0  the 


expression 


J(P2)  =  k0(l-k0/a)P02  +  a2/(l-k0/a)2P02  +  o2  , 


which  yields  P*0  as  in  the  statement  of  the  theorem. 

(ii)  The  LHS  inequality: 

With  the  encoder-decoder  policy  fixed  as  given,  the  problem  faced  by  the  jammer  is 

max  E[(p  2-<*)/32(y)  +  2p*(p*P*o-l)0(y)u] . 


Under  the  assumption  p  2— a<0,  its  solution  is  unique  (because  of  strict  concavity),  and  is 


given  by 


0(y )  =  [p*Cp*P*0 — l)/(P^0(af — p^2))]  y  =  K*y  . 


With  p*  as  indicated,  this  condition  becomes 


which  implies  and  is  implied  by 


—  A.*(  1  +  A*)  <  oil P*02 


3  2^  .  .2 
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Region  R32 

(i)  The  RHS  inequality: 

With  the  jammer’s  policy  fixed  as  indicated,  the  problem  faced  by  the  encoder-decoder 
pair  is 

min  E((k0-k02/a)y2(u)  +  (5(z)— u)2  -  an*2}  . 
y.S 

Using  Lemma  7.2  as  in  Region  R32,  we  obtain  the  encoder  policy  to  be  linear,  with  the 
corresponding  power  level  obtained  by  minimizing  over  P  2  the  expression 

J(P02)  =  k0(l-k0/a)P02  +  (o2  +  o*.M(l-kJot)2P?+o*+o*] 

which  requires  (by  differentiating  with  respect  to  P02  and  noting  that  the  second-order 
condition  for  a  minimum  is  always  satisfied) 

k0(  1— k0/a)  —  (o*+o*,X  1— k0/a)2/[(  1— k0/a)2P*02+a^+a^,]2  =  0 

which  is  satisfied  (uniquely)  by  P*02  =  1  la.  Furthermore, 

6(z)  =  E[u  I  z]  =  \fa  z  . 
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(ii)  The  LHS  inequality: 

With  the  encoder-decoder  policy  fixed  as  indicated,  the  problem  faced  by  the  jammer  is 


max  E{(p*  —  a)0“(y)  +  2p*(p*P*0—  1  )/8(y )  u}  . 
& 


Since  p*"  =  a  and  p*  =  1/P*0,  this  cost  is  independent  of  3,  and  hence  the  jammer's  policy 
may  be  chosen  as  indicated. 


8 


LV 


$ 


J- ' 


157 


I 


I 


i 


§ 


n: 


5 


.V 


£ 

ft 


VS 

nr 


(6)  GameG33 

Consider  the  game  of  Type  3,  under  Criterion  C3.  The  solution  is  provided  in 


Theorem  7.3  below,  after  introducing  some  notation. 


Preliminary  Notation  for  Theorem  73. 

1  2 

We  decompose  the  parameter  space  into  Regions  R33  and  R33,  as  follows: 


R33:  (P02  1/a)  U  (P02  =  1  la  <  4 a  2) 


R2:  P2  =  1/ a  >  4a2  . 


We  next  introduce  the  scalar  parameter  k*  by 


k*  = 


arg  max  |ct^/[(1+X)2P02+o  2)  -  .  i 


2^2, 


in  R 


33 


1 

2 


in  R 


33 


9 

.  s 

Note  that  k*  €  (—1,0). 

Theorem  13.  Game  G 

y. 

> 

where 

(i)  y*(u)  =  P0u 

(li)  fj.*  is  the  Gaussian  probability  measure  associated  with  the  random  variable 


v  =  0*(  y )  = 


\*v 


in  R 


33 


X*y  +  h*  in  R 


33 


where  n*  ~~  N(0,o  “. )  =  N(0,(  l/4«— ou2)/2) 
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(iii)  S*  is  the  Bayes  estimator  for  u  under  the  least  favorable  distribution  /u*  for  v, 


computed  as 


(P„(l+\*)/[P  2( l+k*f+o  2]) z  in  R* 


8*(z)  =  p*z  = 


(1/P0)z 


Proof.  We  shall  establish  separately  the  validity  of  the  RHS  and  the  LHS  inequalities  of 


(7.18)  in  Regions  R33  and  R323. 


Region  R33 


(i)  The  RHS  inequality: 


With  the  jammer’s  policy  fixed  as  indicated,  the  problem  faced  by  the  encoder-decoder 


pair  is 


min  E{(8(z)-u)2  -  <*X.*2y2(u)}  ,  E[y2(u)]  <  P02  , 
y.s 


where  z  =  ( l+A*)y(u)+w.  Noting  that  \*  €  (—1,0)  it  follows  that 


min  E{(8(z)  —  u)  } 

y .« 


2  2 

under  E[y  (u)]  ^  P0  is  achieved  by  y(u)  =  P  u. 


Now  orA.*2E{y2(u)}  is  also  maximized  when  y(u)  =  P  u  and  therefore  we  obtain  the 


solution  of  the  above  problem  as 


y(u)  =  P0u 


=  E[u  I  z]  =  [P  (  l+A*)/(P;(l+\*f+o  2)J  z 
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Cii)  The  LHS  inequality: 

With  the  encoder-decoder  policies  fixed  as  indicated,  the  problem  faced  by  the  jammer  is 

max  EtCp*2— a)/32(y)  +  2p*(p*P0-l)/3(y)u]  , 

the  unique  solution  to  which  is 

/3*(y)  =  [p^Cp’Po-D/CPoCa-p*2))]  y  =  A*y  , 

provided  that 

p*2  -  a  <  0  , 

or  equivalently 

-A*(l+A*)  <  a2/P02  .  (7.19) 

We  now  show  that  the  inequality  (2)  holds  for  all  P0'  in  R33. 

(a)  First  consider  the  case  PQ2  =  1/a  <  4o  2.  Note  that  under  this  condition  the  equa¬ 
tion  A*2  +  A*  +  o*IP 02  =  0  has  no  real  roots,  which  implies  that 

a*2  +  a*  +  02/Po2>  0  , 

and  hence  — A*(  1+A*)  <  o*l P02. 

(b)  Now,  consider  the  case  P02  ^  1/a.  If  P02  =  1/a  we  know  that  A  satisfies 

[(1+A)2P02  +  a^j2  +  (a^(l+A)/A)P02  =  0  ,  (7.20) 

which  after  some  manipulations  yields 

g(A)  =  (A(l+A)+ a2/P02)(o2/P02  +  (l+A)3/A)  =  0  .  (7.21) 

This  function  is  illustrated  in  Figures  7.4a  and  7.4b.  The  extrema  are  found  at  the  inter¬ 
sections  of  the  horizontal  line  at  aA,  the  necessary  condition  being 


r 
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a  =  — a  ^(  1  +\  )/  [A(  ( 1 +A )2P02  +  a l)]2  A  f(A)  . 

The  quadratic  term  of  (7.21)  corresponds  to  the  maximizing  roots,  whereas  the  cubic  term 
in  (7.21)  has  only  one  real  root  which  is  the  minimizing  solution. 

Let 

A(X)  A  X(l+X)  -f  o^l?2  (with  roots  XA  and  X2A), 
and 

B(X)  A  o2/P2  +  (1+X)3/X  (with  real  root  XB), 
so  that  Equation  (7.21)  may  be  rewritten  as 

A(X)  •  B(X)  =  0  . 

From  Figure  7.4  it  is  apparent  that  if  a  >  1  / P02 ,  we  get  a  maximizing  root  smaller  than 
XA,  which  satisfies  (7.19),  and  if  a  <  IIP2,  we  get  a  maximizing  root  larger  than  XA, 
which  satisfies  (7.19),  and  thus  the  proof  for  Region  R33  is  complete. 

Region  R323 

(i)  The  RHS  inequality: 


>  .  / 

7 

) 
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With  the  jammer’s  policy  as  indicated,  the  problem  faced  by  the  encoder-decoder  pair  is 


min  E{(S(z)— u)2  —  aX*2y2(u)  —  a o2.\  ,  E[y2(u)]  ^  P 2 


with  z  =  ( 1+A*)y(u)  +  w  +  r\* 


It  follows  then,  as  in  the  proof  for  Region  R33,  that  the  required  solution  is 


y(u)  =  P0  u 


8(z)  =  E[u  I  z]  =  ( 1/P.)  z  . 
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(ii)  The  LHS  inequality; 

J(y*,8*,/3)  <  J(y*,8*,/3*)  . 

With  the  encoder-decoder  policies  fixed  as  indicated  we  have  p*P0  =  1  and  p*2  =  a.  There¬ 
fore,  the  maximization  problem  faced  by  the  jammer,  i.e., 

max  E[(p*2-a)02(y)  +  2 p*( p*P0—  1 ) j8( y )u]  , 

8(y) 

has  a  cost  function  which  is  independent  of  /3  which  may,  therefore,  be  chosen  as  in  the 
statement  of  the  theorem. 

Remark  7.1.  This  solution  for  the  jamming  policy  in  Region  R33  is  not  unique.  In  fact, 
any  pair  (X*,<7  ,)  which  satisfies 

(1+A.*)2  —  (l+\*)  +  aCa^+o,^.)  =  0 

provides  a  solution,  since  this  gives 

p*  =  (l+\*)P0/((  l+\*)2P02+a  2+o  2.)  =  1/P0  . 

thus  yielding  a  cost  which  is  independent  of  )3.  The  specific  choice  of  the  solution  pair  in 

the  statement  of  the  theorem  is  motivated  by  continuity  of  k*  at  the  boundary  of  R33  and 

R  2 

k33. 

(c)  Game  G34 

It  turns  out  that  Game  G34  does  not  admit  a  saddle-point  solution  and  hence  the  gen¬ 
eral  approach  used  so  far  is  not  applicable.  This  is  because  the  maximin  value  of  the  prob¬ 
lem  is  not  well ‘defined.  (In  order  to  find  the  maximin  value,  we  need  to  fix  the  jammer's 
policy  and  then  minimize  over  the  encoder-decoder  policy;  but  the  jamming  policy  itself 
depends  on  the  encoder  policy  ihrough  the  constraint,  thus  leading  to  dl-posedness. ) 


ft 


WS 


s 
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However,  we  can  still  obtain  a  minimax  solution  by  proceeding  in  two  stages.  First, 
we  formulate  a  related  problem  with  an  enlarged  information  structure  for  which  we  can 
show  the  existence  of  a  saddle-point  solution.  We  then  show  that  the  minimax  value  of 
the  original  problem  and  that  of  the  problem  with  the  enlarged  information  structure  are 
identical,  and  hence  the  saddle-point  solution  f ound  corresponds  to  the  minimax  solution 
desired. 

Towards  this  end.  suppose  that  in  addition  to  having  access  to  the  random  variable 
y  =  y(u),  the  jammer  also  has  access  to  the  policy  y,  in  other  words,  the  jammer’s  policy 
3  belongs  to  the  space  of  all  Borel  measurable  mappings  which  are  functions  of  both  y  and 
y,  i.e.,  v  =  3(y,y).  The  saddle-point  solution  to  the  new  problem  with  this  enlarged 
information  structure  is  presented  below  in  Proposition  7.1  after  introducing  some  nota¬ 
tion. 


Preliminary  notation  for  Proposition  7.1. 
Let 


where 


P*  =  arg  min  f(P) 


p„  £  o 


f(P0)  = 


K?o  +  o'/[(P0-Pj)2+o*]  if  P0  >  P J 


KK  +  1 


if  PQ  <  P, 


Note  that  either  P*0  =  0  or  P*0  >  Pr 

Furthermore,  decompose  the  parameter  space  into  Regions  R34  and  R324  characterized 


5$ 


r;4:  p*0  >  p, 

p*o  =  0  • 


Proposition  7.1.  The  problem  with  the  enlarged  information  structure,  just  described, 
admits  two  saddle-point  solutions  (y*,8*1(u*)  and  (— y*,— 8*,/**)  where 

(i)  y*(u)  =  P*0  u 

(ii)  /jl*  is  the  Gaussian  probability  measure  associated  with  the  random  variable 


”  (pj/||y|Py  inR34 


v  =  /3*(y,y)  = 


in  R2 


(iii)  8*(z)  =  p*  z 


I(P*0-PJ)/((P*0-PJ)2+a^]z  in  R34 


Proof.  We  shall  establish  separately  the  validity  of  the  RHS  and  the  LHS  inequalities  of 
(7.18)  in  R34  and  R34.  (The  key  observation  here  is  that  the  jammer  must  know  his 
opponent’s  policy  in  order  to  determine  a  minimax  solution.  The  enlarged  information 
structure  enables  the  jammer  to  use  the  policies  as  in  (ii)  above.) 


Region  R34. 

(i)  The  RHS  inequality: 

The  problem  faced  by  the  encoder-decoder  pair,  with  the  jammer's  policy  fixed  as 
indicated,  is 


i-v  ,N 

a  >. 
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min  E[k0y2(u)  +  (8(z)— u)2] 
y.5 


z  =  (l  -Pj/||y|py +w  . 

Let  us  first  consider  the  Problem  PI: 

min  E{(8(z)-u)2}  ;  E[y2(u)]  <  P2  . 

y.8 

We  assert  that  the  minimum  value  of  this  problem  is  achieved  by  some  y  satisfying 
E[y2(u)]  =  P02.  To  verify  this,  we  first  assume  the  contrary,  that  the  minimum  is 
achieved  by  a  y  satisfying  E[y2(u)]  =  P2,  where  0  <  Pj  <  P  <  P0.  The  solution  to  the 
Problem  P2: 

min  E{(8(z)  — u)2}  ;  E[y2Cu)]  =  P2  , 

y.5 

is  then  the  same  as  that  for  the  Problem  PI.  However,  for  Problem  P2  we  know  that 

z  =  (1—  Pj/P)y(u)  +  w  . 

By  defining  y'(u)  =  ( 1— Pj/P)y(u),  and  using  Lemma  7.3,  we  find  that  the  solution  to 
Problem  P2  is  linear  and  yields  an  optimal  performance  of 

o  2/((P~Pj)2+a^)  . 

By  using  a  linear  solution  y  =  P0  u  for  Problem  PI,  we  obtain  the  performance 

a2/((P-P;)2+o2) 

whicn  is  cieaily  lower  than  the  one  obtained  for  Problem  P2,  implying  that  the  minimum 
cannot  be  achieved  for  E[y2(u)]  =  P2  <  P02.  We  now  have 


min  E[k0y2(u)  4-  (8(z)  —  u)2] 

y.8 

<£lvJ(«)l  -  P1  <  P,S> 
o  i 

>  k0P02  +  a  2/[(P e-Pj)2+a  2] 

^  min  (k0P02  +  a  2/[(P0-P;)2+a  2)] 
p.>Pj 

=  k0P*02  +  a  2/[(P*0-Pj)2+a  2] 

where  P*0  >  P}  necessarily  exists  in  R34.  With  y(u)  =  P*0  u  and  8(z)  =  E[u  I  z]  the  above 
inequalities  are  all  tight,  implying  that  the  minimum  is  indeed  achieved  by  the  policies 
given  in  the  statement  of  the  theorem. 

(ii)  The  LHS  inequality: 

With  y*  and  8*  fixed  as  indicated,  the  problem  faced  by  the  jammer  is 

max  {p*2/32(y)  +  2p*(p*P*0—  l)/3(y)  u}  , 

0 

subject  to  the  constraint  E[/32(y)]  <  Pf.  Clearly  the  optimal  policy  for  the  jammer  is  to 
use  a  function,  linear  in  u,  at  the  maximum  permissible  power  level,  which,  under  the 
enlarged  information  structure,  is 

/Hy.y)  =  —  CPj/(|y(p  y  . 

Region  R32 

(i)  The  RHS  inequality: 

With  0*(y)  =  —  y,  the  problem  faced  by  the  encoder-decoder  pair  is 

min  E[k0y‘(u)  +  l]  , 

y,6 


the  solution  to  which  clearly  requires  y(u)  =  0  and 
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8(z)  =  E[ulz]  =  0 


Cii)  The  LHS  inequality: 


With  the  encoder-decoder  policies  fixed  at  zero,  the  cost  becomes  independent  of  the 
jammer’s  policy,  who  may  now  use  3(y)  =  — y  as  the  policy  which  yields  the  saddle-point 
solution. 


We  finally  argue  that  the  minimax  values  of  the  two  problems  with  and  without  the 
enlarged  information  structure  are  the  same.  With  y  fixed,  let  3  belong  to  the  space  of 
Borel  measurable  functions  of  y  satisfying  E[/32(y)]  ^  Pj2,  and  let  3  be  in  the  space  of 
Borel  measurable  functions  of  y  and  y  satisfying  E[/32(y,y)]  <  P  2.  Then 


min  max  J(y,8,3)  =  min  max  J(y,8,3)  , 
y.s  $  y.S  S 


since  for  fixed  y  and  8  the  two  inner  maximization  problems  are  identical. 


We  therefore  have  Theorem  7.4  below. 


Theorem  7.4.  Game  G34  admits  a  minimax  solution  which  is  equivalent  to  the  saddle-point 
solution  given  in  Proposition  7.1. 

□ 


7.5.  Minimax  and  Maximin  Strategies  for  Games  G22,  G23  and  G24 


7.5.1.  Minimax  strategies 


If  the  encoding  policy  y  is  restricted  to  be  deterministic,  then  to  every  jamming  pol¬ 
icy  3  that  is  a  function  of  the  encoder  output  y(u),  there  corresponds  a  jamming  policy 
3  =  3  •  y  which  represents  the  same  random  variable 


v  =  3(u)  =  3(y(u)) . 


(7.22) 
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Since  y  is  not  necessarily  invertible,  we  have  the  inequality 


sup  J(y,8,/3(y(u))  <  sup  J(y,8,j3(u)) 
8  ? 


(7.23) 


for  every  pair  (y,8),  where  y  is  restricted  to  be  deterministic.  Taking  the  infimum  of  both 


sides  over  (y,8)  we  obtain  for  k-2,3,4, 


J3k  =  inf  sup  Jk(y,8,/3(y(u))  <  J2k 

(y.S)  a 


(7.24) 


where  J3k  is  the  upper  value  of  a  similar  game  with  fidelity  criterion  Ck,  with  the 


difference  that  in  this  new  game  the  jammer  has  access  to  the  output  of  the  encoder  rather 


than  the  input.  Thus  the  upper  value  of  a  game  of  Type  2  is  bounded  from  below  by  the 


upper  value  of  the  corresponding  game  of  Type  3.  We  shall  next  show  that  this  inequal¬ 


ity  is  in  fact  an  equality,  and  provide  for  each  k€{2,3,4},  a  set  of  minimax  strategies  that 


achieve  this  value. 


Preliminary  notation  for  Theorem  7.5 


(a)  Game  G-, 


Define  Regions  R,2,  R22  and  R22  by 


^22  :  a  ^  k0 


R22  :  a  >  k0  and  <*k0— k02  <  aJo  2 


(7.25) 


R22  :  a  >  k0  and  c*k0—  k0~  ^  a3o^  . 
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2|  Now  define  P*0  and  X*  by 


1 

1 


0 


P*o  = 


(Max{0,aw(Ck0~1-a_1)^-ow)/((l-k0/a)2)})^ 

a 


in  R22 

inR222 

in  R22 


(7.26) 
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X*  = 


-1 

-k0/a 


in  R22 

in  R22  and  R22 


Let  (y*  5*  $*)  be  defined  by 


y*(u)  =  P*0u  in  R22,  R22  and  R23 


(7.27) 


(7.28) 


(P*0(  1+X*)/(P*02(  i+\*)2+0 

> 

in  R22  and  R22 

5*(z)  =  p*z  = 

(1/P*0)z 

in  R22 

(7.29) 

0 

in 

R22 

Mu)  = 

— (  k0/or)P*0U 

in 

R22 

(7.30) 

in 

r22 

-( — )p*0u+t>* 

a 

where  v*  is  an  independent  Gaussian  random  variable,  p*  ~  N(0,k0/cT— k02/a3— a  “). 
(b)  Game  G23 

—  1  —  2 

Define  Regions  R23  and  R;J  by 


•WW.V 
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Ri  :  (P02*-)  U  (P02=-  <  4 a  2) 
a  a 


R2  •  P  2  =  —  >  4a  2 
k23  •  ro  ^  w 


a 


Now  define  X*  by 


X*  = 


arg  max  (o^/((l+X)2P02+a^)— aX2P02) 


in  Ri 


1 

2 


in  R23 


Let  (y*  5*  j5*)  be  defined  by 


y*(u)  =  P0u  in  R23  and  R 23 


S*  = 


(P0(l+X*)/(P02(l+X*)2+o  2))z  in  R2: 

(UP0k  inR 


/3*  = 


X*P0u 


in  R23 


X*P0u+t?* 


in  R23 


23 

2 


23 


2  Vi 

where  t)*  is  an  independent  Gaussian  random  variabie,  r;*  ~  N(0,l/4a— aw)  ) 


( c )  Game  G 


24 


Let 


P*0  =  arg  min  f(P0) 


where 


(7.31) 


(7.32) 


(7.33) 


(7.34) 


(7.35) 


(7.36) 


$ 


3 


koP02+a  */((P0-Pj)2+O  if  P0  >  P; 


f(P0)  =  2 

Vo 


ifP0<Pj 


Note  that  either  P*0  >  Pj  or  P*0  =  0. 


Define  Regions  R24  and  R224  by 


:  P»o  >  P, 

R224  -P*o  =  0- 


Let  (y*  S* /?*)  be  defined  by 


5*(u)  = 


y*(u)  =  P*0u 


C C p*0— P] ) / c C P*0— p J )2+0)z  in  R24 


in  R2 


-P,u 


3*(u)  = 


in  R24 


(7.37) 


(7.38) 


(7.39) 


(7.40) 


(7.41) 


Theorem  7.5.  Sets  of  minimax  solutions  for  games  ^22’  ^23  and  G24  are  provided  by 
(y*  S* /3*)  and  (— y*,— 5*,— 0*)  with  y*,5*  and  13*  defined  above  in  the  various  regions  under 
each  of  the  three  fidelity  criteria. 


Proof.  We  shall  first  establish  that 


J2k  ^  sup  Jk(y*  S*,(3(  u))  =  Jk(y*,5*,0*)  =  J, 

B 


7.42) 


K 


u 
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The  first  inequality  in  (7.42)  is  immediate  because 

I2k  <  sup  Jk(y,8,/3(u)) 

8 

for  all  pairs  (y,3).  J3K  is  the  minimax  value  for  the  corresponding  game  of  Type  3,  and 
we  shall  prove  the  equality  in  (7.42)  by  considering  individually  the  various  regions 
under  each  of  the  criteria  and  comparing  with  the  minimax  strategies  for  problems  of 
Type  3  available  from  Section  7.4. 


(a)  GameG22 
(i)  Region  R22 


Therefore.  /3*(u)  =  0  and 


J2(y*5*,/3)  =  E[1— *02(u)]. 


sup  J2(y*,S* 0(u))  =  J32  in  R22 


(ii)  Region  R22 


The  problem  faced  by  the  jammer  is 


max  EUp*2— o;)/32(u)+2p*( p*P*0— 1  )0(u)u] 


for  which  the  solution  is 


0*(u)  =  — (  k0/«)P*0u  . 


Noting  that  this  provides  the  same  value  as  the  minimax  vai  e  for  Game  G,,  in  Region 


RG,.  we  have 


sup  J,(y*8*./3(  u  ))  =  J,,  in  R,",  . 


(iii)  Region  R22 


The  problem  faced  by  the  jammer  is 


max  E[(  p*2— or )/32(  u  )+2p*(  p*P*0—  1  )/3(  u)u] , 


and  the  cost  is  independent  of  (3,  the  value  being  the  same  as  the  minimax  value  for  Game 

G32  ^22’  *'e,> 

sup  J2(y*,5*  0(u))  =  J32  in  R22  . 

£ 

We  remark  that  while  0  may  be  chosen  arbitrarily  to  satisfy  the  equality,  the  specific 
choice  of  0*  as  indicated  provides  the  saddle- point  solution  for  Game  G23  and  further  pro¬ 
vides  continuity  of  the  jamming  policy  at  the  boundary  of  Regions  R22  and  R22. 


(b)  Game  G23 
(i)  Region  R23 


The  problem  faced  by  the  jammer  is 


max  EfCp*2— a)02(u)-f2p*(p*Po—  l)0(u)u] , 


the  solution  to  which  is 

p*(p*P0-l) 

0*(u)  = - u  =  A*P0u  . 

(a— p*2) 

Since  this  solution  provides  the  same  value  as  the  minimax  strategy  for  G33  we  have 

sup  J,(y*,§*0( u))  =  J33  in  R,j  . 
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(ii)  Region  R 23 

The  problem  faced  by  the  jammer  is  to  minimize  a  cost  function  which  is  indepen¬ 
dent  of  0  as  in  the  case  of  Region  R232  and  we  have 

sup  J3(y*,5*  0(u))  =  J33  in  R23  . 

& 

Again  0  may  be  chosen  arbitrarily;  the  specific  choice  here  is  due  to  the  same  reasons  as 
remarked  for  Region  R22. 


(c)  Game  G-, 


(0  Region  R24 


The  problem  faced  by  the  jammer  is 


max  E{p*202(u)+2p*(p*P*o—l)0(u)u} 


subject  to  E[02(u)]  <  P/. 

Clearly,  the  optimal  policy  for  the  jammer  is  to  use  a  linear  function  of  u  at  the 
maximum  permissible  power  level,  which  gives 

0*(u)  =  PjU 

and  provides  the  same  value  as  the  minimax  strategy  for  Game  G34,  implying  that 

sup  J4(y*  H*  0(  u))  =  J34  in  R24  . 


3 

i 


(ii)  Region  K,4 

With  the  encoding-decoding  policy  pair  fixed  at  zero,  the  cost  is  independent  of  the 


u 

, 

•/i 


jammer’s  policy  which  may  be  chosen  arbitrarily  to  provide 


w 


sup  J4(y*  5»,/3(u))  =  J34  in  R34  • 
& 


We  now  see  that  in  all  cases  we  have 


sup  Jk(y*,5*,/3(u))  =  J3k  , 


and  thus  (7.42)  holds. 


Using  (7.42)  along  with  (7.24)  it  follows  that  J2k  =  for  k€  {2,3,4}  and  that  the 
strategy  sets  given  are  sets  of  minimax  policies. 


7.5.2.  Maximin  Strategies 


Since  fe  D  Ted  we  have 


inf  Jk(y,8,$)  ^  inf  Jk(y,8,0) 
verier,  yer^.ser, 


(7.43) 


for  all  /3€  17.  Taking  the  supremum  of  both  sides  over  £€17,  we  get 


-ilk  ^-ijk  ’ 


(7.44) 


i.e.,  the  maximin  value  for  a  game  of  Type  2  is  bounded  from  below  by  the  maximin 
value  of  the  corresponding  game  of  Type  1  (which  is  also  the  saddle-point  value  of  the 
game  of  Type  1). 


Preliminary  notation  for  Theorem  7.6 


(a)  Game  G, 


Define  Regions  R^,  R ,22  and  R^  as  in  (7.10),  and  define  (P*0,P*j)  by  (7.1 1 ). 
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(b)  Game  G ^ 

Define  Regions  R13  and  R13  as  in  (7.12)  and  define  (P*0,P*j)  by  (7.13). 

(c)  Game  G24 

Define  Regions  R14  and  R14  as  in  (7.14)  and  define  (P*0,P*j)  by  (7.15). 

Theorem  7.6.  Sets  of  maximin  solutions  for  games  G22,  G23  and  G24  are  provided  by 
and  (“X* where 

X*(u)  =  P*0u  (7.45a) 

8*  z)  =  (P*0/(P*02+P*/+O)z  (7.45b) 

and 

/3*(u)  =  T)*  (7.45c) 

where  tj*  is  a  zero  mean  Gaussian  random  variable  with  variance  P*/  which  is  indepen¬ 

dent  of  u  and  w. 

Proof.  Note  that 

=  sup  inf  Jk(y,5,/3)  ^  sup  inf  Jk(y,S,j3) 

^er(ky€r^  fler/1  >ertkl  (7.46) 

fier*  serkl 

where  and  F^1  are  the  sets  of  all  permissible  linear  policies  for  the  encoder  and  the 
decoder,  respectively,  and  the  inequality  follows  because  the  infimum  on  the  right  side  is 
taken  over  a  smaller  set.  We  next  show  that  the  expression  on  the  right  side  of  the  ine¬ 
quality  in  (7.46)  is  J_,k,  i.e.,  the  maxiinin  (or  saddle-point)  value  for  the  corresponding 
game  of  Type  1 . 
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(a)  Game  G, 


Let  y(u)  =  €u  and  S(z)  =  Az.  Then 


J2(y,8,j8)  =  ( 1— A€  )2  +  A2o  2  +  A2E[02(u)] 


+  2A(  €  A— 1  )E[u0(u)]  -  aE[0  (u)]  +  k06  2  . 


Consider  the  minimization  of  the  above  functional  over  (€  ,A).  Using - |A,  =  0  and 

0A 


noting  that - >  0,  we  obtain 

0A2 


€+E[u0(u)] 


€  2+E[02(u)]+  a  >2  €  Etu0(  u)] 


as  the  unique  minimizing  A*  for  every  fixed  € .  Therefore, 


,  E[02(u)]+a2— (E[u0(u)D2 

J2L.  =  Ko€  + - «e[02Cu)]. 

€  2+E[02C  u  )]+  a  1+2  €  E[u0(  u )] 


We  next  minimize  J2|A,  with  respect  to  €.  In  particular,  if  E[u0(u)]  =  0  we  get 


€*z  =  Max  0, 


E[02(u)]+a  2 


-  (E[/3  (u)]+cO 


and,  in  general,  we  have 


=  ko^*Z  + 


E[02(  u)]+  a  *—(  E[u0( u)])2 
- <*E[02(u)] 


e  *2+E[02(  u  )]+  a  1+2  €  *E[u/3(  u )] 


and  we  note  that  sgn(€*)  =  — sgnE[u/3(u)]. 


To  maximize  J|A,  e.  as  a  function  of  E[u0(u)]  we  differentiate  with  respeci  to 


E[u0(u)]  and  find  that 


SW5 


,91  ,1-  <’*  _Stl  k.* 


d(E[u/3(u)]) 


>0  if  E[u/3(u)]  <  0 


d4.,€. 


d(E[u/3(u)]) 


<0  if  E[u/3(u)]  >  0  . 


Therefore,  J|A,a,,  which  is  continuous  in  E[u/3(u)],  admits  its  maximum  at  E[u/3(u)]  =  0 


and  we  next  maximize  over  E[p  (u)]  to  find  that 


E[/32(u)]  =  Max{0  ,  —  -  (S^+a  2)}  . 


Note  that  if  €*  =  0,  then  E[/32(u)]  is  necessarily  zero,  which  is  the  case  if  —  <  a2  i.e., 


Region  R12. 


Next  note  that  €*  could  be  nonzero  with  E[/32 (u)]  =  0,  which  occurs  if 


S*2  =  -  —  a  2 


along  with 


—  <  (€*  +  a 2)2 


which  together  yield 


<  o  2  <  — 


(k0W  ko 


i.e.,  we  have  Region  R,",, 
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Finally  note  that  in  Region  R12  where 


a^(k0  +  a)2  <  k0  , 


both  €  *  and  E[j32(u)]  are  nonzero. 


Observing  that  the  jammer's  maximization  problem  is  solved  by  using  any  second- 


order  random  variable  with  the  appropriate  variance,  it  follows  that  a  Gaussian  random 


variable  as  indicated  is  an  appropriate  choice.  Thus,  with  the  encoding-decoding  policy 


restricted  to  be  linear,  the  cost  obtained  by  using  the  jammer’s  maximinimizing  policy  is 


the  same  as  the  maximin  value  of  Game  G.„ 


(b)  Game  G, 


With  y(u)  =  €  u  and  8(z)  =  Az 


Jj(y,S,/3)  =  ( 1— A€  )2  +  A2a  2  +  A2E[02(u)] 
+  2A(  €  A— 1  )E[u/3(u)]  —  aE[/32(u)] 


and  we  obtain  as  bef  ore 


E[/32(u)]+a  2-(E[u/3(u)]l 


J3  a«  — 


-  aE[p  (u)] 


t  +E[02(u)]+o  2+2€ E[u0(u)] 


and  minimization  of  J|A,  with  respect  to  €  yields 


P0  or  -P0 


if  E[u0(u)]  >  0 
if  E[u0fu)]  <  0 
if  E[u3(u)]  =  0 


(Recall  that  we  require  I  €2  I  ^  P02).  Now, 


-  E[02(u)]+o2-(E[u/3(u)l)2 

P02+E[/32(u)]+a2-2P0E[u/3(u)] 


-  aE[p(u)]  if  E[u/3(u)]  <  0 


E[j32(u)j+o  ^-(E[uj3(u)]) 

'  P02 +E[/32(  u)]+  a  2+2PqE[u/3(  u )] 


—  aE[02(u)]  if  E[u/3(u)]  >  0 


Noting  that  the  derivative  of  this  expression  with  respect  to  E[u0(u)]  is  increasing  for 
negative  E[u0(u)]  and  decreasing  for  positive  E[u/3(u)],  we  find  that  the  unique  maximum 
is  at  E[u0(u)]  =  0. 

To  further  maximize  with  respect  to  E[02(u)]  we  take  the  derivative  and  get 


,  P0  2  2 


E[0z(u)]  =  —  -  (P02  +  o')  if  (— )  >  (P02  +  a  2) 

'/i  lA 

a  a 

(i.e..  Region  R,^),  and  zero  otherwise  (i.e..  Region  R/3). 

Also  note  that  with  the  encoding-decoding  policy  restricted  to  be  linear,  the  cost 
obtained  by  using  the  jammer's  maximizing  policy  is  the  same  as  the  maximin  value  for 
Game  G13. 


(c)  Game  G1 


With  y(u)  =  € u  and  8(z)  =  A z 


J-iIa*  ~  + 


El01  u )]+  a  "-( E[u0(  u )])“ 


€2+E[/32(  u)]+a^+2£  E[u/3(  u)] 
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E[02(u)]+ct  2-(E[u0(u)])2 

^4Ia»,6»  =  ^0^  ■*" 

€  *2+E[32(u)]+  o  2+ 2  €  *E[u/3(u)] 

with  sgn(  €  *)  =  — sgn  E[u/3(u)]. 

We  differentiate  J4|A,  6,  with  respect  to  E[u|3(u)]  to  find  that  the  maximizing  value  is 
E[u/3(u)]  =  0.  Further  since  J4|£l,  e,  is  an  increasing  function  of  E[u0(u)],  its  maximum 
over  0(u)  subject  to  E[u/3(u)]  ^  Pj2  is  attained  at  E[02(u)]  =  P j2.  Any  second-order  ran¬ 
dom  variable  with  the  appropriate  variance  may  be  used  as  the  maximinimizing  solution, 
yielding  a  cost  which  is  the  same  as  the  maximin  value  for  Game  G14. 

We  have  thus  shown  that  for  each  of  the  games  the  expression  on  the  right  side  of 
the  equality  in  (7.46)  is  i.e.,  we  have 

(7.47) 

Now  using  (7.47),  along  with  (7.45),  we  find  that  =J4K,  and  the  strategies  indicated 
are  indeed  the  maximin  strategies. 

□ 

7.6.  Conclusion 

In  this  chapter  general  classes  of  communication  games  have  been  studied,  which 
include  situations  where  the  encoder  mapping  may  be  random  or  restricted  to  be  deter¬ 
ministic,  the  jammer  may  tap  the  input  to  the  encoder  or  the  input  to  the  channel,  and  the 
power  constraints  on  ihe  encoder  and  the  jammer  may  be  hard  or  soft.  For  each  of  the 
above  cases,  minimax,  maximin  or  saddle-point  solutions  have  been  provided. 


The  important  results  obtained  are 


**fTAmuuuHE>K«i  mJt  w*  U!xmn*wv. 


(l)  Games  of  Type  1  where  the  encoder  structure  is  allowed  to  be  probabilistic  and  the 


jammer  taps  the  input  to  the  encoder,  admit  saddle-point  solutions  under  all  fidelity 


criteria  considered. 


(ii)  Games  of  Type  2  where  the  encoder  structure  is  restricted  to  be  deterministic  do  not 


admit  a  saddle-point  solution;  however,  both  minimax  and  maximin  strategies  may 


be  found  in  this  case. 


(iii)  The  minimax  value  for  a  game  of  Type  2  coincides  with  the  minimax  value  of  the 


corresponding  game  of  Type  3,  and  the  maximin  value  for  a  game  of  Type  2  coincides 


with  the  maximin  (saddle-point)  value  of  the  corresponding  game  of  Type  1. 


The  least  favorable  jamming  noise  for  games  of  Type  1  is  Gaussian,  and  this  adds  to 


the  list  of  results  previously  available  in  the  literature  where  the  Gaussian  distribution 


has  been  shown  to  be  extremal.  However,  it  is  important  to  note  that  for  games  of  Type  1 


the  least  favorable  jamming  noise  is  independent  of  the  input,  whereas  for  games  of  Type 


3  this  least  f avorable  noise  is  correlated  with  the  output  of  the  encoder. 


It  is  noteworthy  that  the  analysis  here  cannot  be  trivially  extended  when  the  input 


sequence  is  vector  valued  or  the  number  of  channels  is  more  than  one,  since  the  counter¬ 


part  of  the  standard  Gaussian  test  channel  does  not  admit  a  simple  linear  coding  scheme  in 


the  vector  case. 
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CHAPTER  8 

RECAPITULATION  AND  CONCLUSIONS 

In  this  thesis  we  considered  the  problem  of  simultaneously  designing  communication 
and  control  strategies  for  decentralized  systems.  The  main  thrust  was  towards  identify¬ 
ing  classes  of  problems  with  linear  dynamics,  quadratic  loss  functionals  and  Gaussian 
statistics  for  which  the  optimality  of  linear  strategies  could  be  established.  The  general 
approach  used  consisted  of  first  finding  a  lower  bound  on  the  cost,  and  then  constructing 
strategies  which  achieved  this  lower  bound.  For  some  instances  of  the  cases  in  which 
linear  strategies  failed  to  provide  globally  optimal  solutions,  explicit  nonlinear  strategies 
were  obtained  to  demonstrate  the  inferiority  of  linear  designs. 

In  Chapter  1  we  introduced  scenarios  in  which  the  simultaneous  design  of  communi¬ 
cation  strategies  and  control  policies  may  be  desired,  and  provided  a  discussion  of  various 
types  of  information  structures  associated  with  stochastic  team  problems.  Some  issues  of 
computational  complexity,  which  arise  when  the  numerical  derivation  of  the  optimal 
team  solution  is  attempted,  were  also  discussed  in  Chapter  1. 

In  Chapter  2  we  formulated  and  analyzed  some  fundamental  classes  of  stochastic 
team  problems  with  two  decentralized  agents.  We  identified  those  instances  of  the  general 
problem  for  which  the  optimal  solutions  are  linear.  It  was  shown  that  if  the  first  agent 
observes  an  uncorrupted  version  of  the  variable  to  be  transmitted,  or  if  the  channel  noise 
is  uncorrelated  with  the  input,  then  the  decentralized  team  problem  admits  an  optimal 
solution  which  is  linear  in  the  observation  variables,  and  the  linear  coefficients  mav  be 
found  by  solving  for  the  roots  of  the  fifth-order  polynomial.  For  some  instances  where 
the  first  agent  observes  a  noise-corrupted  version  of  the  variable  to  be  transmitted,  with 
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the  channel  noise  also  correlated  with  this  variable,  we  provided  nonlinear  strategies  that 
outperform  the  optimal  linear  strategies.  We  also  commented  on  some  aspects  of  the 
difficulties  associated  with  Witsenhausen's  problem,  which  is  one  of  the  most  important 
and  most  referenced  counterexamples  in  stochastic  control,  refuting  the  common  belief 
(prior  to  1968)  that  all  linear  quadratic  Gaussian  control  problems  admit  linear  solutions. 

In  Chapter  3  we  considered  stochastic  dynamic  team  problems  where  at  each  step 
two  consecutive  decisions  must  be  taken,  one  being  what  information-bearing  signal  to 
transmit,  and  the  other  regarding  what  control  action  to  exert.  Such  problems  arise  in  the 
simultaneous  optimization  of  both  the  observation  and  the  control  sequences  in  stochastic 
systems.  We  solved  the  problem  completely  for  first-order  systems  under  quadratic  cost 
criteria.  This  was  done  by  first  constructing  an  equivalent  problem  having  a  cost  f unction 
consisting  of  a  sum  of  squared  differences,  and  then  solving  this  equivalent  problem  by 
using  some  bounds  from  Information  Theory.  For  cases  with  hard  power  constraints,  it 
was  shown  that  the  optimum  measurement  strategy  is  to  linearly  amplify  the  innovation 
at  each  stage  to  the  maximum  permissible  power  level.  For  cases  with  soft  power  con¬ 
straints,  the  structure  of  the  solution  was  found  to  be  similar,  with  the  optimum  power 
levels  being  found  via  solving  a  nonlinear  optimal  control  problem,  this  in  turn  being 
done  by  using  a  dynamic  program.  The  results  were  then  extended  to  cases  with  an 
infinite  time  horizon  and  a  discounted  cost  functional,  and  the  existence  of  optimal  sta¬ 
tionary  policies  for  these  problems  was  established. 

In  Chapter  4  we  considered  stochastic  dynamic  decision  problems  requiring  simul¬ 
taneous  optimization  of  both  the  observation  and  the  control  sequences  for  second-  and 
higher-order  systems  under  quadratic  cost  criteria.  We  showed  that  for  some  of  the  sim 


be  outperformed  by  appropriately  chosen  nonlinear  strategies.  We  considered  optimality 
over  the  affine  class  for  problems  involving  a  general  j-th  order  model,  and  showed  that 
within  this  class,  the  optimal  strategy  for  the  hard  power  constraint  problem  consists  of 
transmitting  the  innovation  linearly,  at  each  stage,  at  the  maximum  permissible  power 
level.  For  the  soft  power  constraint  version  the  structure  of  the  solution  was  found  to  be 
similar,  with  the  optimal  power  levels  being  found  via  solving  a  nonlinear  optimal  control 
problem. 

In  Chapter  5  we  generalized  the  results  on  decentralized,  two-person  teams  obtained 
in  Chapter  2,  by  allowing  the  action  of  one  agent  to  be  transmitted  to  the  other  agent 
through  a  number  of  noisy  channels  simultaneously,  instead  of  being  transmitted  through 
a  :  ngle  noisy  channel.  We  showed  that  if  all  channel  noises  are  independent  of  the  input 
variable,  then  linear  strategies  are  optimal,  even  if  the  first  agent  observes  a  noise  cor¬ 
rupted  version  of  the  input,  the  linear  coefficients  being  found  through  a  related  parameter 
optimization  problem.  We  further  showed  that  when  the  channel  noises  are  indeed  corre¬ 
lated  with  the  input  variable,  then  there  are  instances  in  which  the  strategies  which  are 
optimal  over  the  affine  class  may  be  outperformed  by  nonlinear  strategies,  even  when  the 
first  agent  observes  an  uncorrupted  version  of  the  input  variable. 

In  Chapter  6,  the  results  of  Chapter  2  were  generalized  to  cases  with  more  than  two 
decision  makers.  We  considered  problems  with  a  single  transmitting  agent  and  multiple 
receiving  agents,  problems  wiih  multiple  transmitting  as  well  as  receiving  agents,  and 
finally  problems  with  multiple  transmitting  agenis  and  a  single  receiving  agent.  For  prob¬ 
lems  involving  a  single  transmitting  agent  and  multiple  receiving  agents,  we  showed  that 
ihe  optimal  strategies  are  linear  when  either  the  first  agent  observes  an  uncorrupted  ver¬ 
sion  of  the  input,  or  when  all  channel  noises  are  independent  of  the  input.  For  problems 


with  multiple  transmitting  as  well  as  receiving  agents  we  again  found  that  the  optimal 
strategies  are  linear  when  either  all  transmitting  agents  observe  an  vncorrupted  version  of 
the  variable  to  be  transmitted  or  when  all  channel  noises  are  independent  of  this  variable. 
However,  for  the  simplest  classes  of  problems  involving  multiple  transmitting  agents  and 
a  single  receiving  agent,  we  found  instances  in  which  the  strategies  which  are  optimal  over 
the  affine  class  may  be  outperformed  by  appropriately  chosen  nonlinear  strategies,  even 
when  the  first  agent  observes  an  uncorrupted  version  of  the  input  to  be  transmitted,  and 
the  channel  noises  are  independent  of  the  input.  For  this  case,  we  provided  strategies 
which  are  optimal  within  the  affine  class. 

In  Chapter  7,  we  allowed  incomplete  statistical  description  of  the  channel  used  to 
transmit  measurements  between  the  decentralized  agents,  and  sought  optimal  solutions 
under  a  worst-case  scenario.  Assuming  the  unknown  part  of  the  channel  noise  to  be  con¬ 
trolled  by  an  adversary  or  "jammer,”  we  viewed  the  problems  as  zero-sum  games.  We 
considered  a  number  of  cases  depending  on  whether  there  were  hard  power  constraints  or 
soft  power  constraints  on  the  decision  variables.  The  unknown  channel  noise  was  allowed 
to  be  correlated  with  either  the  input  or  the  output  of  the  encoder,  i.e.,  the  jammer  was 
assumed  to  have  the  ability  to  tap  the  channel.  We  found  that  if  the  encoder  is  proba¬ 
bilistic  and  the  jammer  taps  the  input  to  the  encoder,  then  saddle-point  solutions  exist 
under  all  fidelity  criteria.  If,  however,  Ihe  encoder  structure  is  restricted  to  be  deter¬ 
ministic,  then  saddle-point  solutions  do  not  exist,  but  both  maximin  and  minimax  stra¬ 
tegies  may  be  found.  The  maximin  value  here  was  found  to  be  the  same  as  the  maximin 
value  of  a  corresponding  game  where  the  encoder  is  allowed  to  be  random,  with  the  jam¬ 
mer  still  lapping  the  input  to  the  encoder:  and  the  minimax  value  here  was  found  to  be 
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be  outperformed  by  appropriately  chosen  nonlinear  strategies.  We  considered  optimality 
over  the  affine  class  for  problems  involving  a  general  j-th  order  model,  and  showed  that 
within  this  class,  the  optimal  strategy  for  the  hard  power  constraint  problem  consists  of 
transmitting  the  innovation  linearly,  at  each  stage,  at  the  maximum  permissible  power 
level.  For  the  soft  power  constraint  version  the  structure  of  the  solution  was  found  to  be 
similar,  with  the  optimal  power  levels  being  found  via  solving  a  nonlinear  optimal  control 
problem. 

In  Chapter  5  we  generalized  the  results  on  decentralized,  two-person  teams  obtained 
in  Chapter  2,  by  allowing  the  action  of  one  agent  to  be  transmitted  to  the  other  agent 
through  a  number  of  noisy  channels  simultaneously,  instead  of  being  transmitted  through 
a  single  noisy  channel.  We  showed  that  if  all  channel  noises  are  independent  of  the  input 
variable,  then  linear  strategies  are  optimal,  even  if  the  first  agent  observes  a  noise  cor¬ 
rupted  version  of  the  input,  the  linear  coefficients  being  found  through  a  related  parameter 
optimization  problem.  We  further  showed  that  when  the  channel  noises  are  indeed  corre¬ 
lated  with  the  input  variable,  then  there  are  instances  in  which  the  strategies  which  are 
optimal  over  the  affine  class  may  be  outperformed  by  nonlinear  strategies,  even  when  the 
first  agent  observes  an  uncorrupted  version  of  the  input  variable. 
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In  Chapter  6,  the  results  of  Chapter  2  were  generalized  to  cases  with  more  than  two 
decision  makers.  We  considered  problems  with  a  single  transmitting  agent  and  multiple 
receiving  agents,  problems  with  multiple  transmitting  as  well  as  receiving  agents,  and 
finally  problems  with  multiple  transmitting  agents  and  a  single  receiving  agent.  For  prob¬ 
lems  involving  a  single  transmitting  agent  and  multiple  receiving  agents,  we  showed  that 
the  optimal  strategies  are  linear  when  eilher  the  first  agent  observes  an  uncorrupted  ver¬ 
sion  of  the  input,  or  when  all  channel  noises  are  independent  of  the  input.  For  problems 
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the  same  as  the  minimax  value  of  a  corresponding  game  where  the  jammer  taps  the  output 
of  the  encoder. 

The  problems  studied  in  this  thesis  can  be  viewed  as  important  prototype  problems, 
which  could  be  considered  essential  building  blocks  f or  a  general  theory  of  multistage  dis¬ 
tributed  decision  making  under  nonclassical  information,  and  possibly  partial  statistical 
description.  A  major  focus  has  been  on  the  question:  Are  all  stochastic  team  problems 
which  involve  simultaneous  communication  and  control,  and  hence  exhibit  nonclassical 
information  patterns,  inherently  difficult  and  complex?  We  have  shed  some  light  on  this 
question,  obtained  fundamental  results  for  two-stage  stochastic  teams,  and  have  made 
some  contributions  towards  the  development  of  a  general  theory  for  multistage  (finite  and 
infinite  horizon)  stochastic  control  and  team  problems  with  nonclassical  information, 
where  the  control  (decision)  variable  affects  not  only  the  state  trajectory  but  also  the 
quality  of  information  available  to  the  decision  makers. 
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APPENDIX  A 

REFORMULATION  OF  THE  TEAM  PROBLEM 


In  this  appendix  we  show  how  the  squares  may  be  completed  to  obtain  the  equivalent 
problems  PS  and  PH  from  the  problems  PS0  and  PH*,  respectively. 


We  first  note  that  for  k>  i  we  may  write 

i  k-1 


i—  1  n 

xkxi  JL  ®k,i,mXm  Z  Pk,i,n,iVnXi  Z  Pk.i,r,mVnXm^ 

m=0  n*i  n*0  m=n— (j— 2) 


where  the  S’s  are  defined  recursively  as  follows: 


^ioo  —  Pio 


m— 1 


^m,0,0  —  Pm.O  +  T.  Pm,k\.0,0  ^or  1  1 


k=l 

k-1  m— 1 


^k.m.p  Pk.n^n.m.p  £  Pk.n^m.n.p  ^or  P  ^ 


m 


n=m+l  n=p 

k-1 


®k,m.m  Pk.m  Z  Pk.n^n.m.n 


n=m+l 


and  the  p’s  are  defined  by 


Pk.m.k— l,m  ^ 


n-1 


Pk.m.k— n.m  =  I  Pk.k-iPk-i.m.k-n.m  for  2^n^k“m 

i=l 


k— 1  m— 1 

Pk.m.n.i  2-  Pk.tPl.m.n.i  £  Pk.tPm.t.n.i 


t=m+l 


’=0 


We  define  pa  b  c  d  to  be  zero  whenever  any  of  the  following  conditions  holds: 
Ci)  i  <  0 
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(ii)  d  >  Min(b,c) 


(ill)  d  -  b  and  c  >  k-1 


(iv)  c  <  b  and  d  <  c  -  ( j-2) 


and  8t  b  c  is  defined  to  be  zero  whenever  c  >  b. 
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We  therefore  have 


(ajX;  +  5i_1xi_1+...+ai.(j_1)xi_(j_1))2 

i— 1  k  i 

=  Z  2ri+i.k.mxm)  +  Z  ql+ iX\ 

k«0  m«k-(j-2)  k=0 


where  the  r’s  and  q’s  are  given  as  follows: 


V 

V 
$ 


$ 

*/ 

Y 

V 

Vi! 


ri+l.n.p  Z  Z  akamPk,m.n,p  ’ 

k=*i-(j— 2)  m=i — (j — 1) 


<W=  Z  Z  akam5k.m.nfor0^n^i-j 

k=i— (j— 2)  m=i-(j— 1) 


t*l> 

& 

ii* 


^i+l,n  an  Z  Z  akam^k,m,n  " 

k*i— (j— 2)  m=i— (j— 1) 


8 

& 
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We  can  now  write 


(  Z  Pi+  l.nXJ  =  Z  vk(  Z  2si+l.k.mXm)+  Z 


n=i— (j— 1) 


k=0  m=k-(j-2) 


i 
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ai(  Z  bUxa)  =  Z  vk<  Z  2s'i+1  k  mXm)  +  Z  t'i+1.mxm  • 

n»i— (j— 1)  k*0  m=k-(j— 2)  m»*0 

Here  si+1  k4n  and  ti+ln  are  given  by  expressions  identical  to  those  for  rj+1  km  and  qi+ln, 
respectively,  with  the  an  being  replaced  by  pi+1  n,  and  s'i+l  km  and  t'i+1>a  are  given  simi¬ 
larly  by  replacing  an  by  bi  n  and  further  multiplying  by  ait 

The  completing  of  the  squares  now  proceeds  as  follows.  At  the  first  step  we  have 

CN+1XN+1  +  ^NVN 
N 

=  ^NVN  +  CN+1^  VN  Z  PN+l,mXm^  +  mN 

m=N— (}— 1) 

N— 1  k  N 

+  Z  Vk(2  Z  SN+1  kmxm)  +  Z  WmXm) 

k*0  m=k-(j— 2)  m=0 

i.e.,  we  can  complete  the  first  square  to  obtain 


aN  —  +  CN+1 


bN,N  “ 


CN+1^N+1,N 


CN+1^N+1  ,N— ( j— 1) 


3N,N-(j-l) 


Coming  to  the  second  step  now,  we  have  the  following  terms  containing  vN_,: 

^N-lVN-l  +  CNXN  +  ^N+l.N- 1  N+1,N^XN 


+  2vn_,(  Z  ^SN+l,N— l,m  S  N  +  l,N-l,m^Xm 
m=N— 1— (j— 2) 

Letting 


C  N  —  CN  +  ^N+1.N  *  N+l.N^ 


we  obtain 


aN-l  “  ^N-l  +  C  N 

and  expanding  we  get  the  following  expressions  for  bN_j  n’s: 

C  N^N ,N— 1  ^ ®N+ 1  ,N—  1  ,N—  1  ®  N+l.N-l.N-1^ 


■'N— 1,N— 1 


aN_ 


N— 1 


C  N^N.N-j+l  (SN+l,N-l.N-j+l~S  N+l.N— 1,N— ;+l  ^ 


JN— 1,N— j+1 


JN— l.N— j 


aN-l 


c  nPn.n-1 

aN— 1 


Moving  on  to  the  third  step,  we  have  the  following  expression  for  terms  containing  vN_2: 

dN-2VN-2  +  cN-lXN-t+^N+l.N-l— 1  N+1.N-1^XN-1  +  ^N.N-l- 1  N.N-1^XN-1 

N— 2 

+  ^vn-2^  T.  ^SN+l,N-2,m~S  N+l,N-2.m^Xm 

m=N— 2— (j— 2) 

N— 2 

+  ^vN-2^  £  ^SN,N— 2,m  S  N,N— 2,m^Xm  * 

m=N— 2— (j—  2) 


Letting 


we  obtain 


C  N-l  —  CN-l  +  ^lN+l.N-l  1  N+1.N-1-*  +  ^N.N-1  1  N.N-l  ^ 
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aN-2  “  C  N— 1  +  ^N-2 

C  N-iPn-1,N-2~^SN+1.N-2,N-2~S  N+l.N-2,N-2^~^SN,N-2,N-2— 8  N,N-2,N-2^ 


N— 2,N— 2 


aN— 2 


■'N-2.N— j 


bN-2.N-j-l  — 


_  c  N-l^N-l,N-j  ^SN+l,N-2.N-j— 8  N+l,N-2.N-p  ^SN,N-2,N-j  S  N,N-2,N-p 


JN— 2 


C  N-l^N-l.N-j-l 


JN-2 


Proceeding  in  a  similar  fashion  we  can  obtain  the  general  expression  for  the  a’s  and 
b’s  as  follows: 


DC 

K 


N  j-1 

EtZ  ai(v~  £  by-**;-*)2]  +  cN 

i*0  k=0 


where  cN  is  a  constant  given  by 

N 

=  Z  c  i+l0m|  +  c  O0®,,  ' 

i=0 

To  complete  the  transformation,  we  now  need  a  redefinition  of  the  v’s.  Towards  this  end, 
we  first  note  that  with  x’s  defined  by 

i-i 

*i+i  =  Z  Pi+l,i-kXi-k  +  mi  • 

k=0 

we  have  for  i-0,1,2,... 


where 


=  Vi  -  vi 


j-1 

*i  =  vi+  Z  Pi+i,i-kWi)' 

k=0 


Theref ore,  we  have 


j-i 

vi "  Z  bu-kVk 

k=0 

j-1 

=  Vi~  Z  V^rk—Vk-l) 

k=0 

j-1  j-1 

=  vi  +  Z  bu-kVk-i  -  Z  bi,-^i-k 

k=0  k=0 

j-1 

=  Z  bi.i-k*i-k 

k=0 


and  hence,  the  new  v’s  are  defined  in  terms  of  the  original  v’s  by 
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APPENDIX  B 

THE  GAUSSIAN  CHANNEL  WITH  SIDE  INFORMATION  AT  THE  DECODER 


Consider  Problem  PI  defined  below  where  the  objective  is  to  simultaneously  design 
the  measurement  policy  y  and  the  control  policy  8  to  minimize  a  quadratic  cost. 


Problem  Pi 


Minimize  J(u,v)  =  E[k0u2  +  (v— x)2] 

y.6 


(B.l) 


where 


u  =  y(x) 
v  =  8(x,,x2) 


(B.2a) 

(B.2b) 


and 


Xj  =  x  +  w, 


x2  =  U  +  W2 


(B.3a) 

(B.3b) 


Here  x,  Wj  and  w2  are  zero  mean,  independent,  Gaussian  random  variables,  with 


variances  ax2,  a  2^  and  a  2^,  respectively. 


To  facilitate  the  study  of  Problem  PI,  we  formulate  a  related  problem,  P2,  which  is 
the  hard  constraint  version  of  Problem  PI,  i.e.,  the  term  E[k0u2]  is  removed  from  the  cost 
functional  and  the  further  stipulation  is  made  that 

E[y2(x)]  <  P2  . 


(B.4 ) 


We  therefore  have  Problem  P2  below. 
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Problem  P2 


Minimize  J2(u,v)  =s  E[(v— x)2]  CBS'* 

y.s 


given  (2),  (3)  and  (4). 


I  Problem  P2  is  represented  schematically  in  Figure  B.l  below.  Note  that  this  problem 

i 

i 

|  may  be  viewed  as  one  of  designing  encoding  and  decoding  strategies  in  the  presence  of  side 

) 

information  at  the  decoder.  If  the  side  information  were  absent,  then  this  would  be  a  spe¬ 
cial  case  of  the  problem  studied  in  Bansal  and  Basar  [1987a],  without  a  product  term 
between  the  decision  variables,  and  the  linear  solution  would  be  optimal.  For  the  above 
i  problem,  however,  the  linear  solution  is  not  optimal  as  we  shall  elucidate  in  the  sequel. 


Encoding-decoding  problems  with  side  information  have  been  the  subject  of  some 
previous  investigations.  The  rate  distortion  for  source  coding  with  side  information  at  the 
decoder  has  been  studied  for  general  sources  in  Wyner  [1978].  It  has  been  shown  that  if 
the  source  and  side  information  are  jointly  Gaussian,  then  the  minimum  rate  (in  the  usual 
Shannon  sense)  required  for  encoding  the  source  at  a  distortion  level  about  d,  is  equal  to 


Figure  B.l.  Schematics  of  Problem  P2. 
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the  rate  which  would  be  required  if  the  encoder  (as  well  as  the  decoder)  had  access  to  this 
side  information.  Since  the  coding  theorem  uses  the  fact  that  long  ergodic  sequences  are 
asymptotically  typical  and  therefore  can  be  encoded  into  sequences  that  have  the  distribu¬ 
tion  which  achieves  capacity,  Wyner’s  result  implies  that  if  block  encoding  and  arbitrary 
delays  were  permissible,  then  the  least  mean  square  error  for  Problem  P2  would  be  the 
same  as  that  for  the  system  depicted  in  Figure  B.2  below.  Note  that  the  only  difference 
between  Figures  B.  1  and  B.2  is  that  in  the  latter  the  encoder  too  has  access  to  the  side 
information. 

The  situations  we  envisage  are  control  theoretic  applications,  where  the  encoder  out¬ 
put  represents  system  measurements  and  the  decoder  has  to  control  the  system  in  real 
time.  Thus,  bounds  which  are  asymptotically  tight  are  of  little  use  in  our  application, 
where  sample  by  sample  transmission  is  required.  Real-time  coding-decoding  problems 
have  also  been  studied  earlier,  notably  in  Walrand  and  Varaiya  [1983],  where  finite 
sources  are  considered  and  feedback  information  has  been  shown  to  be  useful  for  the 


Figure  B.2.  The  encoder  has  side  information  also. 
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causal  encoding-decoding  problem.  In  the  problems  of  interest  to  us  here,  f eedback  is  not 
permitted. 

Wyner’s  [1978]  result  implies  that  if  block  encoding  were  permitted,  then  the  least 
mean  square  error  for  Problem  P2  would  be  the  same  as  in  the  case  illustrated  by  Figure 
B.2.  For  the  situation  depicted  in  Figure  B.2  we  have 


2  ,  2 

1  °x+£S 

KxiXj)  =  —  log( - ) 

2  a2 

W. 


n2.  2 

1  ^  +°wj 

I(x;x'2  I  x,)  <  —  log( - ) . 

2 


(B.6) 


(B.7 ) 


Now  since 


I(x;x,,x'2)  =  I(x;Xj )  +  l(x;x'2  I  x,), 


(B.8) 


we  have 


l(x;xj,x'2)  <  —  logC 
2 


!  ^x2+<JwI)(p2+°»V 


2  2 

0.„  0„, 


(B.9) 


The  rate  distortion  function  for  a  Gaussian  source  of  variance  ox  under  a  mean 


square  distortion  criterion  is 
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R(d)  =  Max(0,—  log - ) 
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Further, 


Kxix^x'j)  >  I(x;v')  >  —  log- 


(B.1 1) 


E((x— v')2) 


which  implies  that 


2  2  2 


E[(x— v')2]  > 


4  d  . 


(a  2+o  2  XP2+cr  2  ) 


(B.12) 


Now  if  we  use  the  causal  policy 


u(x)  =  X'(x  —  E(x  I  Xj)) 


(B.13) 


where  X'  satisfies 


r>2f  2  ,  2  \ 

,  P^x+°w,) 

(V)2  = - - 


(B.14) 


(  2  2  x 

(®x°w,) 


then  this  least  possible  distortion  d  is  attained  with  equality,  implying  that  the  linear 
strategy  indicated  is  optimal  for  the  problem  with  feedback. 


In  Problem  P2  feedback  is  not  available,  and  hence  the  optimal  linear  strategy  con¬ 
sists  of  using 


u(x) =  Xx 


CB.  15a) 


v  =  E[x  I  x,  ,x2] 


(B.  15b) 


which  yields  the  mean  square  distortion  d*  given  by 


d*  = 


2  2  2 
ax°w,0,2 


+oi  (Ov+<T  2  ) 


(B.  10) 
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Clearly  d*>d,  since  P2ax2>0.  This  is  not  unexpected  since  in  the  case  of  d,  addi¬ 
tional  information  was  used  at  the  encoder.  As  discussed  earlier,  if  arbitrarily  large 
delays  are  allowed,  then  the  least  distortion  for  P2  is  also  d,  hence  strategies  may  be  con¬ 
structed  via  block  encoding  which  would  improve  upon  the  case  of  linear  sample  by  sam¬ 
ple  transmission. 

The  important  issue  now  is  whether  a  reduction  in  the  least  mean  square  error  d* 
(obtained  via  linear  coding)  is  possible  without  block  encoding.  We  now  show  that  the 
optimal  causal  linear  strategy  may  be  outperformed  by  a  causal  nonlinear  strategy,  i.e., 
without  block  encoding  and  thus  with  no  delay. 


We  first  provide  an  example  where  the  optimum  linear  strategy  for  Problem  P2  may 
be  outperformed  by  an  appropriately  chosen  nonlinear  policy.  Towards  this  end,  we  pro¬ 
pose  the  following  design: 

(B.  17) 


v  = 


u  =  x  +  €  sgn  x 
(Xj+X2-€)/2  if  x,  >  0 


( B.  1 8 ) 


(x1+xz+e)/2  if  x,  <  0 


We  next  find  the  mean  square  error  using  the  policy  proposed  in  (B.17)  and  (B.18). 
We  have 

\2i  i— r (  i  x  1  ,  f.  s  .  T-fr 


E((x— v)  ]  =  E[(x— v)  I  x,  ^  0]Prob[x,  ^  0]  +  El(x— v)  I  xt  <  0]Prob(x,  <  0] 


=  E((x— v)‘  I  Xj  ^  0]  . 


Now, 


E((  x — v)2 1  x,  ^  0] 


2*  ‘J'Ss  n  i .  £  S5S  SJ553  £££  Li  g  _  t 


=  E[(x— v)2  I  X!  ^  0,x  ^  0]Prob[x  ^  0 1  Xj  ^  0] 
+  E[(x— v)2  I  xt  >  0,x  <  0]Prob(x  <  0 1  xt>  0] . 


Since 


E[(x— v)2  I  xt  >  0,x  >  0]  = 


o  2  +a  2 

*,  vi 


and 


we  have 


E(x— v)2  I  Xj  ^  0,x  <  0]  = 


2  i  2 

aw  +aw 
W1  w2  2 

- +  / 


2  ,  2 
aw,+aw. 


E[Cx— v)2  |  Xj  >  0]  = - - - 1-  e2Prob[x  <  0  I  Xj>  0] 


Now  Prob(x  <  0  I  xt>  0)  may  be  easily  computed,  since  the  joint  density  of  x  and  x,  is 
known: 


Px.x.(v)“ 


1 


2*-oxa 


exp 


(x,-x) 


2a  1  2a. 


Further, 


Prob(  x<0IXj^0)=l—  Prob(  x  >  0  I  Xj  >  0) 


Prob(x  >  0.x,  ^  0)  ] 

Prob( x  >  01  x.>  0)  =  - 4 - =  21 

Prob(Xj>0)  1/2 


where 
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I=//Px.xdx  dxl  • 


Letting  =  1  we  get 


*”1  x2  (v — x)^ 

I  =  / - exp( - )dx  f  exp ( - )dy 


0  2 wa. 


2  0 


=  / — i —  exp( — —  )dx  — - —  /  e  1  /2dt 


0  'JlLVO  K 


2  'JTtt  -x 


»  .  _  L 

—  f - exp( — —  )d>(— x)dx 

0  2a  2 


1  _ * 

=  / - exp( - )<K-oxz)dz 

0  \J2tt  2 


Thus,  I  may  be  evaluated  and  the  error  using  nonlinear  policies: 


C  A 

'“'NL  “ 


-  +  €2(1— 21) 


may  be  f ound,  and  hence  the  mean  square  error  using  linear  strategies  and  that  using  this 


nonlinear  strategy  may  now  be  compared. 


We  now  provide  some  examples  where  the  optimum  linear  strategy  for  Problem  P2  is 


outperformed  by  a  nonlinear  strategy  of  the  form  given  by  (B.17)  and  (B.18).  We  then 


illustrate  how  these  results  may  be  used  to  construct  instances  of  Problem  PI  where  a 


nonlinear  strategy  may  be  used  to  outperform  the  optimal  linear  strategy. 


Consider  Problem  P2  with  parameter  values  ax“  =  100.0,  0^  =  0^  =1.0.  Letting 


P=  85.042 3,  we  get  e  =  —  1  and  the  nonlinear  strategy  achieves  a  mean  square  error  of 


0.53172.  The  optimum  linear  strategy  at  the  above  power  level  attains  a  mean  square 


error  of  0.53751,  which  is  inferior  to  that  attained  by  the  nonlinear  policy. 
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Table  B.  1  below  gives  other  instances  of  Problem  P2  where  the  nonlinear  policy  out¬ 
performs  the  optimum  linear  policy,  the  third  column  indicating  the  improvement 
(CL— CNL),  i.e.,  the  difference  between  the  mean  square  errors  for  the  two  cases,  and  the 
last  column  indicating  percentage  improvement  over  the  optimum  linear  design.  In  all  the 
cases  considered  in  Table  B.l,  we  have  used  a  2  =  o  2  =1.0. 


TABLE  B.l.  SOME  INSTANCES  OF  PROBLEM  P2  WHERE  NONLINEAR  STRATEGIES 
OUTPERFORM  OPTIMUM  LINEAR  STRATEGIES. 


—t 

Improvement 

%  change 

4.0 

1.00 

0.001375 

0.24 

6.0 

1.10 

0.001736 

6.0 

0.90 

0.009627 

1.74 

6.0 

0.80 

0.011973 

2.19 

6.0 

0.70 

0.013262 

2.46 

6.0 

0.50 

0.012691 

2.41 

10.0 

0.80 

0.009148 

1.73 

10.0 

0.60 

0.009994 

1.92 

10.0 

0.55 

0.009813 

1.89 

We  now  construct  an  instance  of  Problem  PI  by  finding  k0  which  corresponds  to  the 
appropriate  hard  power  constraint.  Returning  to  the  example  with  <j2=  100.0, 
°w  =°«"  =1-0,  we  note  that  when  the  policies  are  linear,  the  optimum  power  level  is 
given  by 


P*'  =  arg  min 


2  2  2 
a„  a,.,  a... 


k()P2  + 


“>  7  7  ,  7  2  x 

P“U  '  +<T  “  ) 


which  implies  that  k0  and  P*2  satisfy 


100 


(P^+lOl)2 


=  0. 


and  for  the  case  in  consideration  with  P2  =  85.0423,  we  may  let 


k0  = - -  0.0028892  . 

(1 86.0423  )2 

Therefore,  for  Problem  PI  with  parameter  values  k0  =  0.0028892,  a x2  =  100.0, 
°w,  =  aw2  ~  1*0,  the  optimum  cost  over  the  linear  class  is  given  by  (0.0028892  x  85.0423 

+  0.53751  -)  0.78321  whereas  a  nonlinear  policy  of  the  form  (B.17)  and  (B.18)  with 
€=  —  1  attains  0.77742. 
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